NRE committed: PLEASE TEST
miguel
* REQUEST * (All, but especially to extension maintainers)
Please test thoroughly! HEAD should be able to load and run without errors
all(?) extensions that were compiled against 8.6 headers.
* WHAT *
NRE is a modification in the deep guts of Tcl that is somewhat similar to
stackless Python. It massively reduces Tcl's C-stack footprint, and enables new
features: the included ::tcl::unsupported::tailcall for now, more exciting stuff
in the future (but in time for 8.6).
For more info please refer to http://msofer.com:8080/wiki?name=NRE
* COMPAT *
NRE is completely transparent for extensions that use the public or internal Tcl
APIs, as long as they do not mess with calls to TEBC directly. It has been
adapted to deal properly with Itcl and TclOO, which do.
As a previous version was tested against many extensions (see
http://msofer.com:8080/wiki?name=NRE+Compatibility), and tclkits were
succesfully built (thanks go to patthoyts) I am confident that this will work
fine and any problems that might arise will be dealt as bugfixes.
* PERF *
The performance impact is generally very small, and often NRE turns out to be
faster: http://wiki.tcl.tk/nre
Enjoy
Miguel
miguel
> * WHAT *
> NRE is a modification in the deep guts of Tcl that is somewhat similar
> to stackless Python. It massively reduces Tcl's C-stack footprint, and
> enables new features: the included ::tcl::unsupported::tailcall for now,
> more exciting stuff in the future (but in time for 8.6).
That should read "but *not* in time for 8.6"
Colin Macleod
This sounds very promising indeed. In particular the concurrency
possibilities suggested at
http://msofer.com:8080/wiki?name=Concurrency+possibilities look like
they could allow us to do coroutines within a single interpreter.
Then we could combine this with the event system to do un-nested
vwaits, and I/O or delays that yield to another coroutine while
waiting, etc. etc. :-)
Colin.
ZB
> they could allow us to do coroutines within a single interpreter.
Something like in Lua?
--
ZB
Colin Macleod
> Dnia 14.07.2008 Colin Macleod <colin.macl...@tesco.net> napisa³/a:
>
> > they could allow us to do coroutines within a single interpreter.
>
> Something like in Lua?
> --
> ZB
Well I don't know a lot about Lua, but I would imagine so.
Colin.
ZB
>> > they could allow us to do coroutines within a single interpreter.
>>
>> Something like in Lua?
>> --
>> ZB
>
> Well I don't know a lot about Lua, but I would imagine so.
miguel
Mmhhh ... stop giving me ideas, I'll never get to sleep.
Would http://msofer.com:8080/wiki?name=Coroutines be suitable?
ZB
>> http://lua-users.org/wiki/CoroutinesTutorial
>
> Mmhhh ... stop giving me ideas, I'll never get to sleep.
>
> Would http://msofer.com:8080/wiki?name=Coroutines be suitable?
I think so... is it possible (or will be soon) to test it?
--
ZB
miguel
Right now, all that's been done is that wiki page :)
I may have something to test soon (a week or two?); it will initially live in
the fossil repo at http://msofer.com:8080, then (maybe) in HEAD under ::unsupported.
I'll announce here whenever things are running; you and everyone else are
welcome to monitor progress, provide feedback and ideas, etc.
Cheers
Miguel
ZB
OK, waiting (im)patiently.
BTW: does there exist a possibility to have "local" installation of TCL?
I mean: especially for testing, made on the "production" machine - not
"system-wide" (to not interfere with already present libraries), but just
in the user account area only?
As I saw, it was very simple with Lua - but TCL/Tk needs(?) to have its
libraries system-widely available (talking about Linux as host-OS)?
--
ZB
miguel
./configure --prefix=/whatever/you/want
make
make install
Alternatively
./configure --disable-shared
make
(and then run it from the build dir, without installing)
ZB
>> As I saw, it was very simple with Lua - but TCL/Tk needs(?) to have its
>> libraries system-widely available (talking about Linux as host-OS)?
>
> ./configure --prefix=/whatever/you/want
> make
> make install
Yes, there's always "--prefix" - but will it limit availability to account
area only? Never tried it such way - maybe I should have been...
> Alternatively
>
> ./configure --disable-shared
> make
>
> (and then run it from the build dir, without installing)
Right, forgot about "static build" possibility.
--
ZB
Donal K. Fellows
> Yes, there's always "--prefix" - but will it limit availability to account
> area only? Never tried it such way - maybe I should have been...
If you install (as yourself) in somewhere below your home directory, Tcl
works just fine. It's availability will be limited to those people who
can read the place where you put it (as per normal Unix perms). FWIW, I
used Tcl this way for many years; there was a system install, but it was
massively out of date.
Donal.
Larry W. Virden
>
> BTW: does there exist a possibility to have "local" installation of TCL?
> I mean: especially for testing, made on the "production" machine - not
> "system-wide" (to not interfere with already present libraries), but just
> in the user account area only?
An alternative that is pretty nifty is the tclkit version of NRE that
was mentioned - that's a one file version of tcl. That way, you can
test out the tcl without any impact whatsoever to directories, etc.
The --prefix configure and build also works great. I've used this for
years.
Finally, I have also used ActiveTcl, whose installer asks for a
location and only writes into that location (well, along with a
$HOME/.teapot directory).
So there are several alternatives to the default directories.
Note that, in my "production" environment, I install Tcl into a
"tclVersion.Level" directory these days. Originally I was using /usr/
tcl - the problem with that is that it prevented me from updating Tcl
without a lot of headaches, trying to get programs which needed to be
updated changed at the same time as I installed the new version of
Tcl. Installing a new version of Tcl into its own original directory
meant that I never impacted the existing environment. It also means
that someone has to take explicit steps to use the new Tcl (which can
be a good or bad thing, depending on the circumstances...)
ZB
> So there are several alternatives to the default directories.
Maybe I'm a bit too careful - but on my machine I've got 2 TCL installs
already: one made directly from sources, and Activestate's TEA archive.
So I prefer a bit of exaggeration rather than to make a total mess... but
indeed, it seems, that local install or static/toolkit will be OK.
--
ZB
Colin Macleod
>
> Wouldhttp://msofer.com:8080/wiki?name=Coroutinesbe suitable?
Sounds great!
Once we have these building blocks, I'm thinking we could create
coroutine-enabled ways of waiting for events (time/input/variable
change).
These might look something like:
co::after delay
- calling coroutine pauses for (at least) delay milliseconds and then
continues, while allowing other events to be processed in the
meantime.
co::vwait variable
- calling coroutine waits until the named variable is set and then
continues. Unlike standard vwait, these calls do not nest, ie.
multiple co::vwaits wait in parallel and can be activated in any
order.
co::read channelId
- calling coroutine reads data from specified channel. If no data is
available the coroutine waits for data, while allowing other events to
be processed.
At present, if you want to write a flow-of-control which includes
waiting for events, and you don't want to block processing of other
events, you have to structure your code as multiple event handlers.
Coroutine-enabled event handling like these co::* commands would let
this be written as straight-line code, which could be much more
natural.
Colin.
Scott
> On 17 Jul, 17:29, miguel <mso...@users.sf.net> wrote:
>
>
>
> > Wouldhttp://msofer.com:8080/wiki?name=Coroutinesbesuitable?
>
> Sounds great!
Ditto
> At present, if you want to write a flow-of-control which includes
> waiting for events, and you don't want to block processing of other
> events, you have to structure your code as multiple event handlers.
> Coroutine-enabled event handling like these co::* commands would let
> this be written as straight-line code, which could be much more
> natural.
Agreed. I've had a number of applications over the years that would
have been far more natural to write as a co-routine than as a chain of
event handlers.
But one thing I'm unclear on is how does NRE work in the presence of C
extensions. Can I have a co-routine invoked if there exists some
custom C code somewhere in the (Tcl) callstack
(Tcl_CreateObjCommand()) above the co-routine yield?
Colin Macleod
>
> But one thing I'm unclear on is how does NRE work in the presence of C
> extensions. Can I have a co-routine invoked if there exists some
> custom C code somewhere in the (Tcl) callstack
> (Tcl_CreateObjCommand()) above the co-routine yield?
No, Miguel has a note on his page saying:
A yield is only allowed if the C stack is at the same level
as it was when the coroutine is invoked; otherwise and error
will be returned and the coroutine will be killed.
It is the last condition that makes things iffy ...
Don Porter
> Alternatively
>
> ./configure --disable-shared
> make
>
> (and then run it from the build dir, without installing)
This won't flawlessly work without additional caretaking.
DGP
miguel
Yes ... and no.
You will not be able to yield from within code created by Tcl_CreateObjCommand;
but you will if it was Tcl_NRCreateCommand.
IOW: NRE runs any standard extension as is, but it cannot take advantage of the
new features if it evals stuff in a non-NRE aware way. To understand what it
means to ne NRE-aware, the best I have to offer for now is
http://msofer.com:8080/wiki?name=Exploiting+NRE
http://www.tcl.tk/cgi-bin/tct/tip/322.html
HTH
Miguel
Neil Madden
I still do this on some Solaris machines at university. System tclsh
there is still 8.0p2! (At one point the tclkit I had in my home dir
became very popular with undergrads running amsn).
-- Neil
Donal K. Fellows
> But one thing I'm unclear on is how does NRE work in the presence of C
> extensions. Can I have a co-routine invoked if there exists some
> custom C code somewhere in the (Tcl) callstack
> (Tcl_CreateObjCommand()) above the co-routine yield?
It won't work. But my experience with NRE-enabling the implementation of
TclOO is that it's not difficult to convert as long as your command code
falls into one of two patterns.
1) Doesn't call back into Tcl (other than via traces, which aren't
NRE-enabled at all). In this case, you don't need to do anything.
2) Can be described as custom code wrapped around a call to Tcl_Eval
(or one of its family of friends). The trick here is to arrange for
the code after the invocation back into the interpreter to be
restructured as a registered callback, which isn't actually
difficult.
The other big case is when you're calling back into the interpreter
multiple times. This might be working now; I've not reviewed all the
past 24 hours' changes yet. :-)
Donal.
Alexandre Ferrieux
TCL_LIBRARY=../library export TCL_LIBRARY
./tclsh86
works for me.
-Alex
Don Porter
>>> Alternatively
>>> ./configure --disable-shared
>>> make
>>> (and then run it from the build dir, without installing)
>> This won't flawlessly work without additional caretaking.
> TCL_LIBRARY=../library export TCL_LIBRARY
> ./tclsh86
>
> works for me.
In my book, a requirement to set a special value for
TCL_LIBRARY counts as additional caretaking.
DGP
Alexandre Ferrieux
Right. I just wanted to say it was minimal caretaking, which was not
obvious from your wording :-)
-Alex
john roll
I just downloaded the cvs HEAD but the comands arn't included:
% info commands ::tcl::unsupported::*
::tcl::unsupported::disassemble ::tcl::unsupported::tailcall
???
miguel
HEAD (as of an hour ago) has in ::tcl::unsupported
disassemble
tailcall
atProcExit
coroutine
yield
Maybe you are linking to the wrong library? A good way to test is to configure
with --disable-shared, which insures that no such snafus happen. Another way is
to go all the may to 'make install', or for quick tests try out 'make shell'.
In order to use them without the ::tcl::unsupported prefix you can do one of
(a) namespace path ::tcl::unsupported
(b) namespace eval ::tcl::unsupported namespace export *
namespace import ::tcl::unsupported::*
(c) interp alias {} tailcall {} ::tcl::unsupported::tailcall
(etc)
john roll
This is a very nice addition to tcl but its way to complex for me to
use. I need some sugar and I'm not sure how to write it. What I want
is a "generator" command which I can use to create different types of
generators, like iota:
generator iota { start end incr } {
for { set i $start } { $i <= $end } { incr $incr } { yield $i }
}
Then I might use it like this:
while { [set x [iota 2 20]] } { puts $x }
With the generator yielding 18 times and then returning "break". I
don't want to know about apply and other functional theory. I also
don't want to have to create a place holder token to hold the
coroutine. I know how I want to use coroutines and I don't want to
know all this other stuff.
Is an API like this possible?
john roll
My fault. I didn't select --disable-shared, Sorry.
John
miguel
mig@cpq:/home/CVS/tcl_SF_clean/unix$ cat /tmp/iota.tcl
namespace path ::tcl::unsupported
proc iota { start end } {
yield
for { set i $start } { $i <= $end } { incr i } {
yield $i
}
return -code break
}
proc UNIQUE_NAME {} {return cor[incr COUNTER]}
proc makeIota { start end } {
set cmd [UNIQUE_NAME]
coroutine $cmd iota $start $end
return $cmd
}
set iota_1_5 [makeIota 1 5]
puts *[info command $iota_1_5]*
puts ---
while 1 {
set x [$iota_1_5]
puts $x
}
puts ---
puts *[info command $iota_1_5]*
mig@cpq:/home/CVS/tcl_SF_clean/unix$ ./tclsh /tmp/iota.tcl
*cor1*
---
1
2
3
4
5
---
**
mig@cpq:/home/CVS/tcl_SF_clean/unix$
john roll
miguel,
This is very close and I was trying to formulate something like it
but, well it was hard for me. Is there a reason that you need the
first yield? Why doesn't "coroutine" create the command by default,
why does it wait for the first yield call? Also understand the
reasons for the "iota_1_5" command, as I called it the "token" to hold
the coroutine, but it feels in the way to me (awkward).
Thanks,
John
Donal K. Fellows
Everything's still experimental right now. This is very new after all.
But we can use other features to make this better...
namespace path tcl::unsupported
proc iota {start end} {
for {set i $start} {$i<=$end} {incr i} {yield $i}
return -code break
}
proc UNIQUE_NAME {} {return cor[incr ::COUNTER]}
proc makeIota {start end} {
set cmd [UNIQUE_NAME]
coroutine $cmd apply {args {yield;tailcall {*}$args}} \
iota $start $end
return $cmd
}
I'm not yet 100% satisfied with what I've written, but it does show
how you can wrap the unpleasantness of the current API.
Donal.
john roll
Miguel, Donal,
I'm very appreciative of these new features going in to the core, and
of your efforts. Tcl has such a clean look (to me) that I want to put
my input on the table early. I know that it is quite difficult to
hide some things due to Tcl's lack of syntax. Where other languages
have special constructs or know something special about a keyword, we
just have commands.
I see in the above example that you've moved the initial yield into
the body of the coroutine call to clear it out of the iota routine
proper, but I'm asking the more fundamental question, why have an
initial execution of the script passed to the coroutine command at
all? Why not just build the coroutine command immediately? Will this
feature be useful elsewhere?
When I initially suggested the possible usage :
while { [set x [iota 2 20]] } { puts $x }
I was going for the feeling of Icon, but I realized after I posted
that Tcl doesn't have the special value that Icon calls failure. The
"while { 1 } { set x [$iota] }" expression is almost as good but it is
less suggestive of what is controlling the loop termination.
Thanks,
John
Fredderic
john roll <jo...@rkroll.com> wrote:
> I see in the above example that you've moved the initial yield into
> the body of the coroutine call to clear it out of the iota routine
> proper, but I'm asking the more fundamental question, why have an
> initial execution of the script passed to the coroutine command at
> all? Why not just build the coroutine command immediately? Will this
> feature be useful elsewhere?
I think it makes sense to think of the usage pattern like this:
proc my-coroutine {args} {
initialisation
yield
# main functionality starts here
while {some-condition} {
calculate-value
yield return-value
}
}
In the simple iota example, then, you can read it simply as there being
no initialisation part.
Personally, I think it would make more sense if [yield] would function
regardless of whether the proc was evaluated via [coroutine] or not,
rather than throwing an error. You could still use [coroutine] to
create simultaneous invocations, and unlike a [coroutine] invocation,
the original proc it would essentially "reset" when it completes.
Basically acting exactly as though it was storing internal state in a
private global variable between runs.
Failing that, I think [coroutine] should work exclusively on lambda's
and not [proc]s at all, to highlight the fact that you seemingly can't
really use them in interchanged contexts; a proc with [yield]s won't
work too wekk when invoked directly rather than via construction by
[coroutine], and a proc without [yield]s doesn't make much sense to
invoke through [coroutine]. So you'd take a lambda style
function-in-a-variable, and use [coroutine] to create a command out of
it, which would then be used instead of invoking the lambda via apply.
That's my take on it, anyhow... Other than that little issue, I think
it's pretty damn awesome, and will help make an awful lot of
client-server type code MUCH neater.
--
Fredderic
Longhorn error#4711: TCPA / NGSCB VIOLATION: Microsoft optical mouse
detected penguin patterns on mousepad. Partition scan in progress to
remove offending incompatible products. Reactivate your MS software.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 47 days, 10:49)
miguel
> Personally, I think it would make more sense if [yield] would function
> regardless of whether the proc was evaluated via [coroutine] or not,
> rather than throwing an error. You could still use [coroutine] to
> create simultaneous invocations, and unlike a [coroutine] invocation,
> the original proc it would essentially "reset" when it completes.
> Basically acting exactly as though it was storing internal state in a
> private global variable between runs.
Not sure about that ... when [yield] fails to accomplish what it is designed to
do, I think it is better for it to error out rather than do something else.
> Failing that, I think [coroutine] should work exclusively on lambda's
> and not [proc]s at all, to highlight the fact that you seemingly can't
> really use them in interchanged contexts; a proc with [yield]s won't
> work too wekk when invoked directly rather than via construction by
> [coroutine], and a proc without [yield]s doesn't make much sense to
> invoke through [coroutine]. So you'd take a lambda style
> function-in-a-variable, and use [coroutine] to create a command out of
> it, which would then be used instead of invoking the lambda via apply.
Conceptually (and very much in code too), what
coroutine foo cmd arg1 arg2 arg3
does is:
1. create a new execution environment
2. create a new command foo that resumes running this execEnv
3. swap in the new execEnv as the current execution environment, insuring that
the caller execEnv is swapped back in when this one exits (return or yield)
4. run (in the new env)
uplevel #0 [list cmd arg1 arg2 arg3]
5. if the execEnv returned, delete it together with the associated command foo
Yield just checks that the current environment is not the main one, ie, that we
are executing within an execEnv that can be suspended. It does not know about
procs, lambdas nor anything else. The other thing it checks is that the C stack
is currently at the same level it was when the execEnv was last resumed. This
means for instance that you can't [*] yield from within an [eval]ed or [source]d
script.
Miguel
[*] for now, until TclEvalEx is adapted to NRE or until we start compiling
everything and junk direct evaluation
Alexandre Ferrieux
>
> [*] for now, until TclEvalEx is adapted to NRE or until we start compiling
> everything and junk direct evaluation
By the way, do you have an up-to-date evaluation of the performance
hit of junking direct evaluation ?
-Alex
miguel
Not that I know of. There are at least three different aspects to consider:
(a) time to compile/run vs interpret
(b) mem reqs, especially when sourcing large files
(c) code simplicity
Alexandre Ferrieux
I would assume the overhead in (a) to be acceptable for everything but
corner cases like people sending thousands of individual "set var
value" commands per second over a socket to be [eval]led... What's
unclear to me is how we could reach the certainty that this bold jump
is worth it, while I strongly feel that it would benefit the majority.
For (b) I'm not sure I understand. Sourcing a file doesn't necessarily
mean you compile it as a single unit and then execute: toplevel
statements can be compiled and run one by one.. Or can they ?
For (c) it is quite obvious that simplicity would increase
drastically. Countless bugs about X working differently when compiled
and not, would vanish instantly. Getting rid of code duplication is
also an extremely valuable move for people "diving in", which is my
case right now ;-) So in the long run, the benefit to the community is
unquestionable.
Comments ?
-Alex
miguel
> On Aug 23, 2:04 pm, miguel <mso...@users.sf.net> wrote:
>> Alexandre Ferrieux wrote:
>>> On Aug 22, 2:52 pm, miguel <mso...@users.sf.net> wrote:
>>>> [*] for now, until TclEvalEx is adapted to NRE or until we start compiling
>>>> everything and junk direct evaluation
>>> By the way, do you have an up-to-date evaluation of the performance
>>> hit of junking direct evaluation ?
>> Not that I know of. There are at least three different aspects to consider:
>>
>> (a) time to compile/run vs interpret
>> (b) mem reqs, especially when sourcing large files
>> (c) code simplicity
>
> I would assume the overhead in (a) to be acceptable for everything but
> corner cases like people sending thousands of individual "set var
> value" commands per second over a socket to be [eval]led... What's
> unclear to me is how we could reach the certainty that this bold jump
> is worth it, while I strongly feel that it would benefit the majority.
Not sure about that ...
> For (b) I'm not sure I understand. Sourcing a file doesn't necessarily
> mean you compile it as a single unit and then execute: toplevel
> statements can be compiled and run one by one.. Or can they ?
If you compile them one by one, it will be *very* slow: there is some constant
overhead per compilation, and you'd pay it over and over. I am planning to
experiment with doing it statement by statement, only compiling when there are
nested evals (ie, avoiding recursive evals as long as there are no traces[*]).
Stay tuned ...
OTOH, compiling the whole file may have big advantages in runtime ... if/when I
get around to fully spec'ing and coding "all bytecodes have a local var table,
not just the proc/lambda bodies". Not trivial but feasible.
[*] NRE-enabling traces: not yet considered, will eventually get done.
> For (c) it is quite obvious that simplicity would increase
> drastically. Countless bugs about X working differently when compiled
> and not, would vanish instantly. Getting rid of code duplication is
> also an extremely valuable move for people "diving in", which is my
> case right now ;-) So in the long run, the benefit to the community is
> unquestionable.
Wrong: this has nothing to do with that. Even if you compile all scripts,
something like
set set set
$set x foo
or even
[join [list s e t] {}] x foo
cannot be compiled inline. Or do you mean that the non-compiled Tcl_SetObjCmd
would actually compile the script 'set x foo'? In any case, we do need some
non-compiled version of the command, to be called when an otherwise bytecompiled
command cannot be recognized at compile time.
> Comments ?
>
> -Alex
>
>
>
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> Personally, I think it would make more sense if [yield] would
>> function regardless of whether the proc was evaluated via
>> [coroutine] or not, rather than throwing an error. You could still
>> use [coroutine] to create simultaneous invocations, and unlike a
>> [coroutine] invocation, the original proc it would essentially
>> "reset" when it completes. Basically acting exactly as though it
>> was storing internal state in a private global variable between
>> runs.
> Not sure about that ... when [yield] fails to accomplish what it is
> designed to do, I think it is better for it to error out rather than
> do something else.
How is it not doing what it's designed to do? If it fails because of
an impure stack, then by all means error out. But [yield] would be a
handy way to allow a [proc] to do some initialisation on first run.
The only issue I'm aware of, is how they would mix. [yield] would end
up yielding the current [coroutine] instead of creating a new one. (I
presume in your implementation, you can [yield] from any TCL call
depth.) It would need a [yield -local] switch (or its converse) to
function the way I would like to see it, I guess. I'd like to know
which most people think is the sanest default behaviour if both were
available; yield back to the inner-most [coroutine] and error if there
isn't one, or yield the current proc only which is more what the term
co-routine implies to me. (In either case there would be an option to
perform the other.)
Something that would at least let me fudge what I would like to see, is
the ability for [coroutine] to push the current proc out of the way and
bring it back later. So you'd end up with something like this:
proc something {args} {
coroutine something {{abc def} {
{ ... the real body goes here ... }
}} {*}$args
}
I'll certainly be adding a -coroutine switch to my standard [proc]
wrapper, I think, though it's going to be ugly and flakey without that
kind of capability.
I presume there'll be an [info] sub-command to tell us whether we're in
a [coroutine] or not...?
>> Failing that, I think [coroutine] should work exclusively on
>> lambda's and not [proc]s at all, to highlight the fact that you
>> seemingly can't really use them in interchanged contexts; a proc
>> with [yield]s won't work too wekk when invoked directly rather than
>> via construction by [coroutine], and a proc without [yield]s
>> doesn't make much sense to invoke through [coroutine]. So you'd
>> take a lambda style function-in-a-variable, and use [coroutine] to
>> create a command out of it, which would then be used instead of
>> invoking the lambda via apply.
> Conceptually (and very much in code too), what
> coroutine foo cmd arg1 arg2 arg3
> does is:
> 1. create a new execution environment
Could you elaborate on this part, please? What is different in this
new execution environment compared to the regular one, and how heavy is
this new environment. Would it be feasible to do this as part of the
regular proc invocation process?
By my understanding this basically entails starting a fresh TCL stack,
and pre-loading it with a small hook to regain control later on, within
which you evaluate the subject code.
I suppose my question, is whether there's anything within the TCL stack
that will stop you from basically taking the current TCL call level
with you to this new environment. As I understood it (which may well
be utterly wrong here ;) ), the TCL stack portions were relocatable, it
was (and still is) the C chunks that got in the way, which is what the
NRE eval deals with by allowing the C stack to unwind before performing
the eval.
> 2. create a new command foo that resumes running this execEnv
> 3. swap in the new execEnv as the current execution environment,
> insuring that the caller execEnv is swapped back in when this
> one exits (return or yield)
> 4. run (in the new env) uplevel #0 [list cmd arg1 arg2 arg3]
> 5. if the execEnv returned, delete it together with the associated
> command foo
Steps 2-5 sound like standard library-based task-swapping mechanics.
> Yield just checks that the current environment is not the main one,
> ie, that we are executing within an execEnv that can be suspended. It
> does not know about procs, lambdas nor anything else.
It was the fact that [yield] will work from any level that had slipped
my attention. Still, having [coroutine] take a lambda instead of a
proc would make it nestable within a proc, as in the example up top,
and I do still think it makes a whole lot more sense, as I said, with
[yield] not working from the main environment, even though [yield] can
still occur within other procs.
I wonder whether it would be sane to add an [info lambda] command that
will compile and return a proc as a lambda expression...?
> The other thing it checks is that the C stack is currently at the
> same level it was when the execEnv was last resumed. This means for
> instance that you can't [*] yield from within an [eval]ed or
> [source]d script.
I presume these two can be fixed, though...? Which leaves just
old-style non-NRE-aware extensions.
--
Fredderic
Elephant = mouse built to government specs
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 51 days, 22:22)
Alexandre Ferrieux
> If you compile them one by one, it will be *very* slow: there is some constant
> overhead per compilation, and you'd pay it over and over.
OK; I had wrongly assumed that the overhead was rather linear in the
size of the script, implying that it would be the same whether the
glueing occurs before (whole-script compilation) or after (per-
statement) compilation, which to my superficial understanding is just
the serialization of the AST.
Of course I'll believe you: I'm learning :-)
> > For (c) it is quite obvious that simplicity would increase
> > drastically. Countless bugs about X working differently when compiled
> > and not, would vanish instantly. Getting rid of code duplication is
> > also an extremely valuable move for people "diving in", which is my
> > case right now ;-) So in the long run, the benefit to the community is
> > unquestionable.
>
> Wrong: this has nothing to do with that. Even if you compile all scripts,
> something like
> set set set
> $set x foo
> or even
> [join [list s e t] {}] x foo
> cannot be compiled inline. Or do you mean that the non-compiled Tcl_SetObjCmd
> would actually compile the script 'set x foo'?
Yes I do !
> In any case, we do need some
> non-compiled version of the command, to be called when an otherwise bytecompiled
> command cannot be recognized at compile time.
Again my knowledge of all this may lack depth, but the duplication
between the INST_LIST_INDEX case in TEBC
and the non-compiled Tcl_LindexObjCmd makes me wince. So make one call
the other, acknowledging the overhead. Slowing down Tcl by 1% (random
figure) might be worth eliminating the risk of desync in the
duplicated parts... But maybe the devil is in the actual figure ;-)
-Alex
Donal K. Fellows
> It was the fact that [yield] will work from any level that had slipped
> my attention. Still, having [coroutine] take a lambda instead of a
> proc would make it nestable within a proc, as in the example up top,
> and I do still think it makes a whole lot more sense, as I said, with
> [yield] not working from the main environment, even though [yield] can
> still occur within other procs.
Actually, [coroutine] takes a list of words after the name of the
command to create upon [yield]. The word list can designate a call of
a procedure or an [apply] call if you wish...
Donal.
miguel
> On Fri, 22 Aug 2008 09:52:18 -0300,
> miguel <mso...@users.sf.net> wrote:
>
>> Fredderic wrote:
>>> Personally, I think it would make more sense if [yield] would
>>> function regardless of whether the proc was evaluated via
>>> [coroutine] or not, rather than throwing an error. You could still
>>> use [coroutine] to create simultaneous invocations, and unlike a
>>> [coroutine] invocation, the original proc it would essentially
>>> "reset" when it completes. Basically acting exactly as though it
>>> was storing internal state in a private global variable between
>>> runs.
>> Not sure about that ... when [yield] fails to accomplish what it is
>> designed to do, I think it is better for it to error out rather than
>> do something else.
>
> How is it not doing what it's designed to do? If it fails because of
> an impure stack, then by all means error out. But [yield] would be a
> handy way to allow a [proc] to do some initialisation on first run.
Oh - now I understood what you mean. This would make procs a bit heavier than
coroutines (see below). As currently implemented, it would also prevent the
second and later calls to take more than one argument.
IOW, your proposal may look like "let us change [yield], but ir really is "let
us change [proc]".
>
>
> The only issue I'm aware of, is how they would mix. [yield] would end
> up yielding the current [coroutine] instead of creating a new one. (I
> presume in your implementation, you can [yield] from any TCL call
> depth.) It would need a [yield -local] switch (or its converse) to
> function the way I would like to see it, I guess. I'd like to know
> which most people think is the sanest default behaviour if both were
> available; yield back to the inner-most [coroutine] and error if there
> isn't one, or yield the current proc only which is more what the term
> co-routine implies to me. (In either case there would be an option to
> perform the other.)
I think you are talking about stackful versus non-stackful coroutines, in the
terminology of http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf. The
experimental coroutines in HEAD are stackful, what the term implies to you seems
to be non-stackful.
> Something that would at least let me fudge what I would like to see, is
> the ability for [coroutine] to push the current proc out of the way and
> bring it back later. So you'd end up with something like this:
>
> proc something {args} {
> coroutine something {{abc def} {
> { ... the real body goes here ... }
> }} {*}$args
> }
>
> I'll certainly be adding a -coroutine switch to my standard [proc]
> wrapper, I think, though it's going to be ugly and flakey without that
> kind of capability.
>
> I presume there'll be an [info] sub-command to tell us whether we're in
> a [coroutine] or not...?
What is required for [info] is still an open question. Also a way to query if a
given command is a coroutine or not might be useful. Or if [yield] will succeed.
(as opposed to catching it). I'm waiting to see what the users need.
In the current implementation of CallFrames the previous levels at creation time
cannot be preserved, and that is why the coroutine runs at level #0. Preserving
the complete CallFrame stack is not impossible, but it is not a minor
undertaking as a lot of infrastructure has to change. It may happen for 8.7 or
9.0 if interesting use cases warrant it, but that's not within a reasonable
horizon for 8.6
The Tcl stack portions are not relocatable, no: there are pointers to them being
saved (eg by [upvar] used in child CallFrames, or stuff that is allocated using
TclStacklloc). The change you propose would imply that the Tcl stack goes, that
CallFrames are allocated independently (and refcounted). Something like that may
yet happen during the 8.6 cycle, I have some ideas that would make the cost of
separate allocs disappear. But it would be purely an internal change, just an
enabler for other things that may lie in the future.
>> 2. create a new command foo that resumes running this execEnv
>> 3. swap in the new execEnv as the current execution environment,
>> insuring that the caller execEnv is swapped back in when this
>> one exits (return or yield)
>> 4. run (in the new env) uplevel #0 [list cmd arg1 arg2 arg3]
>> 5. if the execEnv returned, delete it together with the associated
>> command foo
>
> Steps 2-5 sound like standard library-based task-swapping mechanics.
>
>
>> Yield just checks that the current environment is not the main one,
>> ie, that we are executing within an execEnv that can be suspended. It
>> does not know about procs, lambdas nor anything else.
>
> It was the fact that [yield] will work from any level that had slipped
> my attention. Still, having [coroutine] take a lambda instead of a
> proc would make it nestable within a proc, as in the example up top,
> and I do still think it makes a whole lot more sense, as I said, with
> [yield] not working from the main environment, even though [yield] can
> still occur within other procs.
>
> I wonder whether it would be sane to add an [info lambda] command that
> will compile and return a proc as a lambda expression...?
I'm afraid I did not understand much of this. You say "having [coroutine] take a
lambda instead of a proc" [it takes neither, it takes a command to run in the
new env] would make it nestable within a proc [I do not understand what you mean
here]".
>
>> The other thing it checks is that the C stack is currently at the
>> same level it was when the execEnv was last resumed. This means for
>> instance that you can't [*] yield from within an [eval]ed or
>> [source]d script.
>
> I presume these two can be fixed, though...? Which leaves just
> old-style non-NRE-aware extensions.
I hope to be fixing these two during the alpha/beta cycle. I am really worried
about [eval], not so much about [source].
Some other things do preempt yielding, traces come to mind and may yet be fixed
(haven't even explored what it would take).
miguel
> Fredderic wrote: Oh - now I understood what you mean. This would make procs a
> bit heavier than coroutines (see below). As currently implemented, it would
> also prevent the second and later calls to take more than one argument.
Not than coroutines, than currently ... sorry
Donal K. Fellows
> Again my knowledge of all this may lack depth, but the duplication
> between the INST_LIST_INDEX case in TEBC
> and the non-compiled Tcl_LindexObjCmd makes me wince. So make one call
> the other, acknowledging the overhead. Slowing down Tcl by 1% (random
> figure) might be worth eliminating the risk of desync in the
> duplicated parts... But maybe the devil is in the actual figure ;-)
IIRC, this sort of thing is justified by benchmarks (see the tclbench
module in tcllib). I believe that the issue was that calls out of the
bytecode engine itself were surprisingly expensive.
Donal.
Colin Macleod
> On 17 Jul, 17:29, miguel <mso...@users.sf.net> wrote:
>
>
>
> > Wouldhttp://msofer.com:8080/wiki?name=Coroutinesbesuitable?
>
> Sounds great!
>
> Once we have these building blocks, I'm thinking we could create
> coroutine-enabled ways of waiting for events (time/input/variable
> change).
> These might look something like:
>
> co::after delay
> - calling coroutine pauses for (at least) delay milliseconds and then
> continues, while allowing other events to be processed in the
> meantime.
>
> co::vwait variable
> - calling coroutine waits until the named variable is set and then
> continues. Unlike standard vwait, these calls do not nest, ie.
> multiple co::vwaits wait in parallel and can be activated in any
> order.
>
> co::read channelId
> - calling coroutine reads data from specified channel. If no data is
> available the coroutine waits for data, while allowing other events to
> be processed.
>
> At present, if you want to write a flow-of-control which includes
> waiting for events, and you don't want to block processing of other
> events, you have to structure your code as multiple event handlers.
> Coroutine-enabled event handling like these co::* commands would let
> this be written as straight-line code, which could be much more
> natural.
>
> Colin.
I've posted very simple coroutine-enabled versions of after, vwait and
gets at http://wiki.tcl.tk/21555
Colin.
Fredderic
> Fredderic wrote:
>> [yield] would be a handy way to allow a [proc] to do some
>> initialisation on first run.
> Oh - now I understood what you mean. This would make procs a bit
> heavier than coroutines (see below).
I kind of figured that, and pretty much wiped that idea myself.
My wish here would be served by the [coroutine] command being able to
displace an existing proc or command until it completes. Having
[coroutine] displace the current proc with an inline [apply] invocation
(as DKF mentioned is possible), would allow exactly the semantics I was
rather hoping would be possible. A proc could then trivially be built
that can [yield] without having to be explicitly invoked through
[coroutine].
Though perhaps someone will come up with a neat [proc] wrapper that
uses [rename]s to achieve the same goal... Maybe it's to do with being
1:42 in the morning, but it made my head spin trying to nut it out just
now. ;(
> As currently implemented, it would also prevent the second and later
> calls to take more than one argument.
Dare I ask, why? ;)
>> I presume there'll be an [info] sub-command to tell us whether
>> we're in a [coroutine] or not...?
> What is required for [info] is still an open question. Also a way to
> query if a given command is a coroutine or not might be useful. Or if
> [yield] will succeed. (as opposed to catching it). I'm waiting to see
> what the users need.
Having thought about it a little more, I'm wondering how [coroutine]
interacts with things like [after]. Being able to distinguish between
being [after]d or [uplevel #0]d, and being started directly by
[coroutine], is about the only useful piece of [info] I can think of
at this point. A self-generator could avoid self-generating in the
specific case that it was invoked by [coroutine]; an issue that would
be avoided if [proc] was modified as I was getting at originally, to
allow a [yield -local] or something to effectively detatch the current
[proc] as though it had been started with [coroutine] under the same
name. I understand though that doing that would mess with things like
[upvar] in rather nasty ways...
My focus on this issue, btw, is mostly in the interest of allowing
coroutines to be used as an implementation detail in the construction
of a proc, rather than a project-wide design decision from the outset.
I think _if_ it can be done without too much effort, it would be a
fantastically handy way to do the first-run initialisation thing.
>>>> Failing that, I think [coroutine] should work exclusively on
>>>> lambda's and not [proc]s at all, to highlight the fact that you
>>>> seemingly can't really use them in interchanged contexts; a proc
>>>> with [yield]s won't work too well when invoked directly rather
>>>> than via construction by [coroutine], and a proc without [yield]s
>>>> doesn't make much sense to invoke through [coroutine]. So you'd
>>>> take a lambda style function-in-a-variable, and use [coroutine] to
>>>> create a command out of it, which would then be used instead of
>>>> invoking the lambda via apply.
DKF answered this perfectly by pointing out that you can have [apply]
as the command being invoked by [coroutine]. Something like;
proc generator {blah fubar} {
coroutine goober apply {{abc def} {
do something useful
}} $blah $fubar
}
or expanded into my self-generator idea:
proc generator {blah fubar} {
rename generator temp-name
coroutine generator apply {{abc def} {
do something useful
...
rename generator ""
rename temp-name generator
}} $blah $fubar
}
Simplifying that, is where allowing [coroutine] to displace an
existing proc or command would come in handy, allowing the second
[generator] proc to be written as simply as the first one, without all
that [rename] guff and the problems that go with it (such as how to
handle being unexpectedly [rename]d again by something else in between).
> In the current implementation of CallFrames the previous levels at
> creation time cannot be preserved, and that is why the coroutine runs
> at level #0. The Tcl stack portions are not relocatable, no: there
> are pointers to them being saved (eg by [upvar] used in child
> CallFrames, or stuff that is allocated using TclStacklloc).
Fair enough. Makes perfect sense.
> The change you propose would imply that the Tcl stack goes, that
> CallFrames are allocated independently (and refcounted). Something
> like that may yet happen during the 8.6 cycle, I have some ideas that
> would make the cost of separate allocs disappear. But it would be
> purely an internal change, just an enabler for other things that may
> lie in the future.
I think I've read about that type of structure a couple years back,
where each "stack frame" is independent, and can form a tree or graph
layout rather than a linear monolithic stack. I can quite imagine that
would be a fairly major change...
>> I wonder whether it would be sane to add an [info lambda] command
>> that will compile and return a proc as a lambda expression...?
I still think that would be handy, btw... Essentially this;
proc info::lambda {proc} {
set args [list]
foreach arg [info args $proc] {
if { [info default $proc $arg def] } {
lappend args [list $arg $def]
} else {
lappend args $arg
}
}
return [list $args $body]
}
(Probably needs some [namespace] magic too...)
> I'm afraid I did not understand much of this. You say "having
> [coroutine] take a lambda instead of a proc" [it takes neither, it
> takes a command to run in the new env] would make it nestable within
> a proc [I do not understand what you mean here]".
What I was getting at there, was that stuff with the "generator" proc
up above. So all sorted.
--
Fredderic
Only a stupid person doesn't judge. But only an equally stupid
person forgets the limitations of their judgements.
--- Me! circa 2005
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 61 days, 10:05)
miguel
> miguel <mso...@users.sf.net> wrote:
>
>> Fredderic wrote:
>>> [yield] would be a handy way to allow a [proc] to do some
>>> initialisation on first run.
>> Oh - now I understood what you mean. This would make procs a bit
>> heavier than coroutines (see below).
>
> I kind of figured that, and pretty much wiped that idea myself.
>
> My wish here would be served by the [coroutine] command being able to
> displace an existing proc or command until it completes. Having
> [coroutine] displace the current proc with an inline [apply] invocation
> (as DKF mentioned is possible), would allow exactly the semantics I was
> rather hoping would be possible. A proc could then trivially be built
> that can [yield] without having to be explicitly invoked through
> [coroutine].
>
> Though perhaps someone will come up with a neat [proc] wrapper that
> uses [rename]s to achieve the same goal... Maybe it's to do with being
> 1:42 in the morning, but it made my head spin trying to nut it out just
> now. ;(
I did not understand this section, sorry. Now my head is spinning too :P
>> As currently implemented, it would also prevent the second and later
>> calls to take more than one argument.
>
> Dare I ask, why? ;)
Yup: the argument to the coroutine is the return value of the [yield] that
suspended the previous invocation. A design decision - may be good, may be bad.
>>> I presume there'll be an [info] sub-command to tell us whether
>>> we're in a [coroutine] or not...?
>> What is required for [info] is still an open question. Also a way to
>> query if a given command is a coroutine or not might be useful. Or if
>> [yield] will succeed. (as opposed to catching it). I'm waiting to see
>> what the users need.
HEAD now has ::tcl::unsupported::infoCoroutine (will become [info coroutine]
eventually) that returns the FQN of the coroutine's command (and {} when not in
a coroutine; I know, no {}-named coroutines).
Uhh? I think it might make more sense to do things the other way round, actually:
interp alias {} $proc {} ::apply $lambda
Donal K. Fellows
> Uhh? I think it might make more sense to do things the other way round, actually:
> interp alias {} $proc {} ::apply $lambda
That's better actually; it's sane when you [rename] it...
Donal.
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> miguel <mso...@users.sf.net> wrote:
>>> As currently implemented, it would also prevent the second and
>>> later calls to take more than one argument.
>> Dare I ask, why? ;)
> Yup: the argument to the coroutine is the return value of the [yield]
> that suspended the previous invocation. A design decision - may be
> good, may be bad.
I used to think wrongly that [coroutine]s command was always going to
be a proc, and hence you could pass in arguments and have them update
the corresponding variables with each resume. But I think I'm better
now. ;)
Okay, so you give an argument to the coroutine and it pops out as the
return value of [yield]. Sounds about perfect to me. Just one
question; is it just one word, or a list of the words given to the
coroutine...?
>>>> I presume there'll be an [info] sub-command to tell us whether
>>>> we're in a [coroutine] or not...?
>>> What is required for [info] is still an open question. Also a way
>>> to query if a given command is a coroutine or not might be useful.
>>> Or if [yield] will succeed. (as opposed to catching it). I'm
>>> waiting to see what the users need.
> HEAD now has ::tcl::unsupported::infoCoroutine (will become [info
> coroutine] eventually) that returns the FQN of the coroutine's
> command (and {} when not in a coroutine; I know, no {}-named
> coroutines).
Not necessarily... A list containing an empty string is different from
an empty list.
I'm not 100% sure this isn't being discussed in the core group... But I
think it was said that the coroutine runs at global level (#0)...? Is
that a design decision, or a practical limitation? I'm asking because
since you can pass in arguments to the coroutine (a Good Thing!), how is
[upvar] going to react? For example in this type of construct;
coroutine coincr apply {{} {
while { 1 } {
uplevel 1 incr [yield $count] [incr count]
}
}}
will doing [coincr fuya] work? (Ignoring for the moment whether the
return value of [yield] is a single word or an args-style list of
arguments.)
>>>> I wonder whether it would be sane to add an [info lambda] command
>>>> that will compile and return a proc as a lambda expression...?
>> I still think that would be handy, btw... Essentially this;
>> proc info::lambda {proc} {
>> set args [list]
>> foreach arg [info args $proc] {
>> if { [info default $proc $arg def] } {
>> lappend args [list $arg $def]
>> } else {
>> lappend args $arg
>> }
>> }
>> return [list $args $body]
>> }
>> (Probably needs some [namespace] magic too...)
> Uhh? I think it might make more sense to do things the other way
> round, actually: interp alias {} $proc {} ::apply $lambda
That's pretty off topic anyhow, really, it was more a musing because
[info args] is half-way useful, but [info default] is just a pain in the
proverbial buttocks. So I was thinking it'd be nice to have an [info]
command to compliment [info proc], [info body], [info args], and [info
default] by returning a fully-built lambda, retaining the bytecompiled
version of the proc if it exists (assuming doing so is even possible).
Back before I was enlightened (ie. when I wrote that last post) it
seemed a little more useful than it does now. Still, trying to
construct a lamba that matches an existing proc is still a pain in
those buttocks I mentioned just before. [info lamba] could potentially
share everything with the proc it originally came from making it
extremely light-weight and avoiding the need to re-compile it all over
again and again and again. Where it might still be useful, though, is
making it really easy to pass a proc to another thread...
--
Fredderic
Elephant = mouse built to government specs
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 62 days, 2:56)
miguel
> On Tue, 02 Sep 2008 19:41:54 -0300, miguel <mso...@users.sf.net> wrote:
>
>> Fredderic wrote:
>>> miguel <mso...@users.sf.net> wrote:
>>>> As currently implemented, it would also prevent the second and later
>>>> calls to take more than one argument.
>>> Dare I ask, why? ;)
>> Yup: the argument to the coroutine is the return value of the [yield] that
>> suspended the previous invocation. A design decision - may be good, may be
>> bad.
>
> I used to think wrongly that [coroutine]s command was always going to be a
> proc, and hence you could pass in arguments and have them update the
> corresponding variables with each resume. But I think I'm better now. ;)
>
> Okay, so you give an argument to the coroutine and it pops out as the return
> value of [yield]. Sounds about perfect to me. Just one question; is it just
> one word, or a list of the words given to the coroutine...?
Just one ... now, but it could change: I need to hear arguments pro and con.
What I came up with is:
(a) if 1, caller has to [list] to pass several things in there; if n, receiver
has to [lindex [yield] 0] when it expects just one. I somehow expect that the
most usual cases will be 0 (return value {} ignored), then 1, then n. The case 0
is indifferent between the alternatives, of course. Depends on the use cases
really ...
(b) 1 was much easier to code, and suitable for at least the experimental
version, and I had already so much s**t to get right ...
>>>>> I presume there'll be an [info] sub-command to tell us whether we're
>>>>> in a [coroutine] or not...?
>>>> What is required for [info] is still an open question. Also a way to
>>>> query if a given command is a coroutine or not might be useful. Or if
>>>> [yield] will succeed. (as opposed to catching it). I'm waiting to see
>>>> what the users need.
>> HEAD now has ::tcl::unsupported::infoCoroutine (will become [info
>> coroutine] eventually) that returns the FQN of the coroutine's command (and
>> {} when not in a coroutine; I know, no {}-named coroutines).
>
> Not necessarily... A list containing an empty string is different from an
> empty list.
>
> I'm not 100% sure this isn't being discussed in the core group... But I
> think it was said that the coroutine runs at global level (#0)...? Is that a
> design decision, or a practical limitation? I'm asking because since you
> can pass in arguments to the coroutine (a Good Thing!), how is [upvar] going
> to react? For example in this type of construct;
>
> coroutine coincr apply {{} { while { 1 } { uplevel 1 incr [yield $count]
> [incr count] } }}
>
> will doing [coincr fuya] work? (Ignoring for the moment whether the return
> value of [yield] is a single word or an args-style list of arguments.)
Try it out ...: you'll get an 'undefined variable 'count' error on the first
yield. I do not quite understand what you're trying to do though.
As for the coro running in #0 and having no access to its caller's scope: it is
a limitation dictated
(a) by economy of implementation, doing anything else requires deep modification
of the whole CallFrame concept in Tcl's guts (from stack to tree). May be done
in the future, but it is a relatively largeish thing.
(b) by analogy: as one of the main uses of coroutines will be (I expect, may be
wrong) as event callbacks, this does precisely the same thing we've been doing
forever. Callback scripts run in #0.
(c) for semantic simplicity, the alternative may have some sharp edges that need
to be thought out carefuly. This one I understand fully [famous last words]
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> On Tue, 02 Sep 2008 19:41:54 -0300, miguel wrote:
>>> Yup: the argument to the coroutine is the return value of the
>>> [yield] that suspended the previous invocation. A design decision
>>> - may be good, may be bad.
>> Okay, so you give an argument to the coroutine and it pops out as
>> the return value of [yield]. Sounds about perfect to me. Just one
>> question; is it just one word, or a list of the words given to the
>> coroutine...?
> Just one ... now, but it could change: I need to hear arguments pro
> and con. What I came up with is:
>
> (a) if 1, caller has to [list] to pass several things in there; if n,
> receiver has to [lindex [yield] 0] when it expects just one. I
> somehow expect that the most usual cases will be 0 (return value {}
> ignored), then 1, then n. The case 0 is indifferent between the
> alternatives, of course. Depends on the use cases really ...
I'm looking at it from the least WTFish perspective... Commands don't
generally expect you to wrap up all their arguments into a list for
them. It just doesn't seem natural. heh
Besides, if you care about the arguments, you're going to be doing a
whole lot more than just [lindex ... 0], and it'll give me an excuse to
start another round of arguments for {n}$list being equivalent to
[lindex $list n]... ;)
But seriously, constraining the coroutine to one argument means it
simply can't be used where a regular command taking more than one
argument would be, without wrapping it in yet another proc (even
[interp alias] can't fix that one!).
> (b) 1 was much easier to code, and suitable for at least the
> experimental version, and I had already so much s**t to get right ...
heh Now that I can understand. Though I can't imagine it being more
than a few lines...
I can't get at the API docs right now, but there _must_ be a function to
TclList-ify an array (or portion thereof) of Tcl_Obj's...!
>> I think it was said that the coroutine runs at global level
>> (#0)...? Is that a design decision, or a practical limitation?
>> I'm asking because since you can pass in arguments to the coroutine
>> (a Good Thing!), how is [upvar] going to react? For example in
>> this type of construct;
>> coroutine coincr apply {{} { while { 1 } { uplevel 1 incr [yield
>> $count] [incr count] } }}
>> will doing [coincr fuya] work? (Ignoring for the moment whether
>> the return value of [yield] is a single word or an args-style list
>> of arguments.)
> Try it out ...: you'll get an 'undefined variable 'count' error on
> the first yield. I do not quite understand what you're trying to do
> though.
I'm still using the standard 8.5.3 as provided by the Debian repo's...
It's not quite here yet. On the up-side, this is how I see it as
someone relatively new to the implementation, who hasn't been working
on it for months or otherwise knows it backwards and already knows what
not to get confused about. ;)
It's a contrived example, but basically it _should_ increment the first
variable passed by 1, then the next one by 2, then the third by 3,
etc. It's inspired by a piece of code I've seen used that searches for
the next unused index in a series. Using [coroutine] it could be
written as a simple never-ending loop that runs until it hits an unused
index and then [yield]s the index it found. Not entirely sure that the
[coroutine] environment is lighter than a global variable, but that's a
permitted oversight for contrived examples. ;)
One place where I'd seriously like to use [coroutine] in my own toys
when it finally makes it through to Debian's repo, unfortunately makes
fairly heavy use of [upvar], essentially dragging a couple variables
right through practically the entire proc call stack, and needs those
changes to propagate backwards. It's fixable, but ugly.
> As for the coro running in #0 and having no access to its caller's
> scope: it is a limitation dictated
>
> (a) by economy of implementation, doing anything else requires deep
> modification of the whole CallFrame concept in Tcl's guts (from stack
> to tree). May be done in the future, but it is a relatively largeish
> thing.
I was afraid you'd say that. :(
I wonder, any way of allowing [upvar] to reach between environments?
I'm thinking an [info] command that returns an identifier for the
current coroutine environment which can be handed to [upvar] as a level
identifier, and a means to find out the identifier of the previous
environment. (Real references would be better, but that's a
frightening place we've already spent a looooong time at... ;) )
> (b) by analogy: as one of the main uses of coroutines will be (I
> expect, may be wrong) as event callbacks, this does precisely the
> same thing we've been doing forever. Callback scripts run in #0.
Yes, but those things are generally separated by time, so it makes sense
that locals up in the call stack wouldn't be available anymore. There
may well not be a call stack to look up through.
coroutines aren't. The call stack still exists, your [yield] returns
to it. Even still I've hashed this problem out before too, how to
provide access to a variable an unknown number of levels down the call
stack, without the need of any special magic. (My example was a
command whose implementation had grown and was now several proc calls
deep, passing only a simple value to another function which was written
to expect a variable exactly two levels below it, both functions were
old code and their semantics were locked in by other code elsewhere.)
I wonder if that's the key, then. If [upvar] is being asked to reach
below #0, check whether it's a coroutine, and add a little machinery to
be able tor each backwards. At least until something better comes
along. I'm pretty sure [upvar] (and [uplevel] too) already do that
check for error generation.
> (c) for semantic simplicity, the alternative may have some sharp
> edges that need to be thought out carefuly. This one I understand
> fully [famous last words]
hehehe Well, that's why I'm dragging up issues I can see BEFORE it
hits the masses (including myself). DKF got all grumpy at me for not
speaking up about how crummy [dict]s commands were, when it was being
worked on. So I'm making a point of going through this new feature I'm
most certainly going to be using this time.
--
Fredderic
That's odd. That's very odd. Wouldn't you say that's very odd?
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 62 days, 12:30)
miguel
No no, you fail to understand: there is "something" below #0 for the coroutine,
but it might be completely different at each call! If you create the coroutine
in some wrapper proc, for example, the previous caller does not even exist when
you resume. Consider [spawn] in http://wiki.tcl.tk/21532
So: either the coroutine has to rebind all upvared variables at each resume
(can't!), or has them linked to bog knows what (can't either).
Hence: communicate with coroutines via namespace/global vars.
Neil Madden
> miguel <mso...@users.sf.net> wrote:
>
>> Fredderic wrote:
>>> [yield] would be a handy way to allow a [proc] to do some
>>> initialisation on first run.
>> Oh - now I understood what you mean. This would make procs a bit
>> heavier than coroutines (see below).
>
> I kind of figured that, and pretty much wiped that idea myself.
>
> My wish here would be served by the [coroutine] command being able to
> displace an existing proc or command until it completes. Having
> [coroutine] displace the current proc with an inline [apply] invocation
> (as DKF mentioned is possible), would allow exactly the semantics I was
> rather hoping would be possible. A proc could then trivially be built
> that can [yield] without having to be explicitly invoked through
> [coroutine].
A limitation of this approach is that you couldn't have 2 or more
simultaneous coroutine instances based on the same command. Another
difficulty is that the original proc may take different arguments than
the resume command. You should be able to write something like this if
you want, though:
proc coproc {name params body} {
set f [list $params $body]
interp alias {} $name {} coproc:spawn $name $f
}
proc coproc:spawn {name f args} {
coroutine $name ::apply $f {*}$args
}
(restoring the original proc left as an exercise).
>> As currently implemented, it would also prevent the second and later
>> calls to take more than one argument.
>
> Dare I ask, why? ;)
If you think about how the mechanism works, a coroutine suspends from a
yield, and is later resumed from that place. Any arguments to the resume
become the result of the yield as far as the coroutine is concerned:
set next [yield $val]
Therefore, if the resume command could take multiple arguments then it
would be impossible to distinguish between multiple separate arguments
and a list. You could maybe extend yield, something like:
yield $val here are the resume args
but I think the original is sufficient.
[...]
>
> My focus on this issue, btw, is mostly in the interest of allowing
> coroutines to be used as an implementation detail in the construction
> of a proc, rather than a project-wide design decision from the outset.
> I think _if_ it can be done without too much effort, it would be a
> fantastically handy way to do the first-run initialisation thing.
Coroutines should be possible to use as an implementation detail, as in
my worked example converting a simple synchronous program to use the
event loop: http://wiki.tcl.tk/21532
-- Neil
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> I wonder, any way of allowing [upvar] to reach between environments?
>> I'm thinking an [info] command that returns an identifier for the
>> current coroutine environment which can be handed to [upvar] as a
>> level identifier, and a means to find out the identifier of the
>> previous environment. (Real references would be better, but that's
>> a frightening place we've already spent a looooong time at... ;) )
>>> (b) by analogy: as one of the main uses of coroutines will be (I
>>> expect, may be wrong) as event callbacks, this does precisely the
>>> same thing we've been doing forever. Callback scripts run in #0.
>> The call stack still exists, your [yield] returns to it. I wonder
>> if that's the key, then. If [upvar] is being asked to reach below
>> #0, check whether it's a coroutine, and add a little machinery to be
>> able tor each backwards. At least until something better comes
>> along. I'm pretty sure [upvar] (and [uplevel] too) already do that
>> check for error generation.
> No no, you fail to understand: there is "something" below #0 for the
> coroutine, but it might be completely different at each call!
Actually, that's exactly what I did understand. If the argument(s) to
the coroutine are being fed through the [yield], then for [upvar] or
[uplevel] to work with it, it needs to link to whatever is below #0
right now, with the understanding that what's there now may be totally
different to what's there after the next [yield].
> So: either the coroutine has to rebind all upvared variables at each
> resume (can't!), or has them linked to bog knows what (can't either).
Or just let them go; letting the [upvar]d variables disappear on the
next [yield] would be better than nothing, and could be accomplished
with something resembling an unset trace? And I totally agree that
rebinding every variable every time we resume the coroutine is a rather
sucky idea.
Just trying to get a handle on the problem for my own understanding; is
there any difference between a local variable, and an alias constructed
by [upvar]...? I'm assuming from what you wrote that a variable is
essentially a named pointer to a pointer to the value. [upvar] would
then create a new named pointer to that same pointer to the value. So
the break down would happen if that stack frame went away while we
weren't looking. Or am I off with the fairies somewhere, again? ;)
Failing all that, [uplevel] shouldn't suffer from any of those issues,
and could even be used to fudge an inter-coroutine [upvar], especially
if the #level syntax can be extended to address a level within another
coroutine (most likely counting from its #0 upwards). It may even be an
acceptable interim hack to add such fudging into [upvar] to make it
look like it works for consistency sake, albeit painfully slowly.
Just food for insanity... ;)
--
Fredderic
Don't panic.
Parents of young organic lifeforms are warned that towels can be
harmful... if swallowed in large quantities.
-- Vague reference to HHGTTG
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 63 days, 1:25)
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
> Fredderic wrote:
>> miguel <mso...@users.sf.net> wrote:
>>> Fredderic wrote:
>>>> [yield] would be a handy way to allow a [proc] to do some
>>>> initialisation on first run.
>>> Oh - now I understood what you mean. This would make procs a bit
>>> heavier than coroutines (see below).
>> I kind of figured that, and pretty much wiped that idea myself.
>> My wish here would be served by the [coroutine] command being able
>> to displace an existing proc or command until it completes. Having
>> [coroutine] displace the current proc with an inline [apply]
>> invocation (as DKF mentioned is possible), would allow exactly the
>> semantics I was rather hoping would be possible. A proc could then
>> trivially be built that can [yield] without having to be explicitly
>> invoked through [coroutine].
> A limitation of this approach is that you couldn't have 2 or more
> simultaneous coroutine instances based on the same command.
Not at all... The musings I muttered on coroutines ages ago over on
the wiki were along these lines (with the unenlightened assumption that
[yield] would only effect the proc it was invoked from). In those
musings a [coroutine] command would be used to explicitly create a
simultaneous instance until a new name, where the original command
would simply resume the default one. No reason the same couldn't be
applied here.
> Another difficulty is that the original proc may take different
> arguments than the resume command.
Actually that came out rather cleanly in the wash. The original proc
takes initialisation arguments, while the resumed coroutine gets any
arguments via the return value from [yield]. Unless it takes specific
action, those arguments will be ignored in the resuming case anyhow.
>>> As currently implemented, it would also prevent the second and
>>> later calls to take more than one argument.
>> Dare I ask, why? ;)
> If you think about how the mechanism works, a coroutine suspends from
> a yield, and is later resumed from that place. Any arguments to the
> resume become the result of the yield as far as the coroutine is
> concerned: set next [yield $val]
I believe I just said that as the answer to your prior question. ;)
> Therefore, if the resume command could take multiple arguments then
> it would be impossible to distinguish between multiple separate
> arguments and a list. You could maybe extend yield, something like:
No. You simply ALWAYS return a list, ala [proc]s special "args"
argument. It might be slightly inconvenient, but it's an awful lot
more convenient than not being able to do it at all. There are other
possibilities also, such as allowing [yield] to take an arguments list
after the return value. A bit odd, but it's an idea if you're
desperate. ;)
Besides, I'd still love to see {n} be synonymous for [lindex n], which
would make that issue a whole lot less annoying. ;)
>> My focus on this issue, btw, is mostly in the interest of allowing
>> coroutines to be used as an implementation detail in the
>> construction of a proc, rather than a project-wide design decision
>> from the outset. I think _if_ it can be done without too much
>> effort, it would be a fantastically handy way to do the first-run
>> initialisation thing.
> Coroutines should be possible to use as an implementation detail, as
> in my worked example converting a simple synchronous program to use
> the event loop: http://wiki.tcl.tk/21532
It gets a little more complicated when you're not writing the entire
application from scratch. When you're trying to retro-fit new ideas
into old code (which I tend to do a lot of), things get a little more
interesting.
--
Fredderic
Longhorn error#4711: TCPA / NGSCB VIOLATION: Microsoft optical mouse
detected penguin patterns on mousepad. Partition scan in progress to
remove offending incompatible products. Reactivate your MS software.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 63 days, 2:39)
Neil Madden
> On Wed, 03 Sep 2008 20:19:33 +0100,
> Neil Madden <n...@cs.nott.ac.uk> wrote:
>
>> Fredderic wrote:
>>> miguel <mso...@users.sf.net> wrote:
>>>> Fredderic wrote:
>>>>> [yield] would be a handy way to allow a [proc] to do some
>>>>> initialisation on first run.
>>>> Oh - now I understood what you mean. This would make procs a bit
>>>> heavier than coroutines (see below).
>>> I kind of figured that, and pretty much wiped that idea myself.
>>> My wish here would be served by the [coroutine] command being able
>>> to displace an existing proc or command until it completes. Having
>>> [coroutine] displace the current proc with an inline [apply]
>>> invocation (as DKF mentioned is possible), would allow exactly the
>>> semantics I was rather hoping would be possible. A proc could then
>>> trivially be built that can [yield] without having to be explicitly
>>> invoked through [coroutine].
>> A limitation of this approach is that you couldn't have 2 or more
>> simultaneous coroutine instances based on the same command.
>
> Not at all... The musings I muttered on coroutines ages ago over on
> the wiki were along these lines (with the unenlightened assumption that
> [yield] would only effect the proc it was invoked from).
Link?
In those
> musings a [coroutine] command would be used to explicitly create a
> simultaneous instance until a new name, where the original command
> would simply resume the default one. No reason the same couldn't be
> applied here.
If the original command has been replaced by a default coroutine, how do
you construct other coroutines based on it?
>> Another difficulty is that the original proc may take different
>> arguments than the resume command.
>
> Actually that came out rather cleanly in the wash. The original proc
> takes initialisation arguments, while the resumed coroutine gets any
> arguments via the return value from [yield]. Unless it takes specific
> action, those arguments will be ignored in the resuming case anyhow.
Two cases:
1. resume only takes a single argument, whereas the original proc takes
>1, in which case anybody trying to call the original will get an error.
2. resume takes multiple arguments and the coroutine expects >1, whereas
the original proc takes only 1 argument, will again result in an error
for callers of the original proc.
The point is that the original proc and the coroutine resume command can
have wildly different signatures and expectations. Dynamically replacing
one with the other is likely to create incompatibilities in most cases.
What does it achieve?
>
>
>>>> As currently implemented, it would also prevent the second and
>>>> later calls to take more than one argument.
>>> Dare I ask, why? ;)
>> If you think about how the mechanism works, a coroutine suspends from
>> a yield, and is later resumed from that place. Any arguments to the
>> resume become the result of the yield as far as the coroutine is
>> concerned: set next [yield $val]
>
> I believe I just said that as the answer to your prior question. ;)
>
>
>> Therefore, if the resume command could take multiple arguments then
>> it would be impossible to distinguish between multiple separate
>> arguments and a list. You could maybe extend yield, something like:
>
> No. You simply ALWAYS return a list, ala [proc]s special "args"
> argument. It might be slightly inconvenient, but it's an awful lot
> more convenient than not being able to do it at all. There are other
> possibilities also, such as allowing [yield] to take an arguments list
> after the return value. A bit odd, but it's an idea if you're
> desperate. ;)
The alternative is to leave things as they are and pass a list when you
want to pass multiple values. That has the benefit of symmetry.
>
>>> My focus on this issue, btw, is mostly in the interest of allowing
>>> coroutines to be used as an implementation detail in the
>>> construction of a proc, rather than a project-wide design decision
>>> from the outset. I think _if_ it can be done without too much
>>> effort, it would be a fantastically handy way to do the first-run
>>> initialisation thing.
>> Coroutines should be possible to use as an implementation detail, as
>> in my worked example converting a simple synchronous program to use
>> the event loop: http://wiki.tcl.tk/21532
>
> It gets a little more complicated when you're not writing the entire
> application from scratch. When you're trying to retro-fit new ideas
> into old code (which I tend to do a lot of), things get a little more
> interesting.
Did you read the page? The entire point of the example is that you can
rewrite the implementation without touching the main logic. I.e., using
coroutines makes it *easier* to "retrofit new ideas into old code".
-- Neil
miguel
>> So: either the coroutine has to rebind all upvared variables at each resume
>> (can't!), or has them linked to bog knows what (can't either).
>
> Or just let them go; letting the [upvar]d variables disappear on the next
> [yield] would be better than nothing, and could be accomplished with
> something resembling an unset trace? And I totally agree that rebinding
> every variable every time we resume the coroutine is a rather sucky idea.
>
> Just trying to get a handle on the problem for my own understanding; is there
> any difference between a local variable, and an alias constructed by
> [upvar]...? I'm assuming from what you wrote that a variable is essentially
> a named pointer to a pointer to the value. [upvar] would then create a new
> named pointer to that same pointer to the value. So the break down would
> happen if that stack frame went away while we weren't looking. Or am I off
> with the fairies somewhere, again? ;)
Something like that, yes. That is the reason why [upvar] as implemented will not
let you create links as follows:
% proc fail {} {upvar 0 foo ::foo}
% fail
bad variable name "::foo": upvar won't create namespace variable that refers to
procedure variable
As long as things are in the CallFrame stack the deletion order guarantees that
things disappear sanely, that we never get a "dangling pointer".
> Failing all that, [uplevel] shouldn't suffer from any of those issues, and
> could even be used to fudge an inter-coroutine [upvar], especially if the
> #level syntax can be extended to address a level within another coroutine
> (most likely counting from its #0 upwards). It may even be an acceptable
> interim hack to add such fudging into [upvar] to make it look like it works
> for consistency sake, albeit painfully slowly.
Oh - this is not "fudging upvar/uplevel", this is redesigning the whole concept
of levels in Tcl.
BTW: currently yield does nothing but stop execution and shift some structures
out of the way. It does not traverse the variable list to decide what to do with
upvars or anything like that. Just save some data (currently 5 pointers or
integers). As a matter of fact, to the suspended coroutine it just looks as if
it invoked any old proc ... and was put in the freezer until it returns: the
exact same data is saved/restored by NRE in proc-calling-proc situations.
You may want to think of this as follows: coroutines are designed to run
properly when called from the event loop, eg as in http://wiki.tcl.tk/21532.
Anything else that can be done with them is pure gravy. Note that they can
contain themselves a command dispatcher that does whatever you want based on
what [yield] returns, dispatching via [switch], ensembles, an associated object
(TclOO, XOTcl, iTcl, snit, ...), arrays or dicts containing command names,
variable traces, whatever. So the possibilities are endless even if not
exhaustively so, and still largely unexplored.
Although it did take me a bit to get right (and am not sure I'm there yet), the
current model is a relatively simple leveraging of the NRE infrastructure. No
major rewrites were needed, no code was modified except in the command dispatch
and bytecode execution areas.
> Just food for insanity... ;)
Be my guest: little sanity left here anyway. And *I* will not be recoding major
areas of Tcl from scratch to add more WIBNIs. Not this year at least.
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
> Fredderic wrote:
>> On Wed, 03 Sep 2008 20:19:33 +0100,
>> Neil Madden <n...@cs.nott.ac.uk> wrote:
>>> Fredderic wrote:
>>>> My wish here would be served by the [coroutine] command being able
>>>> to displace an existing proc or command until it completes.
>>> A limitation of this approach is that you couldn't have 2 or more
>>> simultaneous coroutine instances based on the same command.
>> Not at all... The musings I muttered on coroutines ages ago over on
>> the wiki were along these lines (with the unenlightened assumption
>> that [yield] would only effect the proc it was invoked from).
> Link?
Urgh... Nope. My browser keeps crashing right back out. Not quite
sure what's going on just yet.
>> In those musings a [coroutine] command would be used to explicitly
>> create a simultaneous instance until a new name, where the original
>> command would simply resume the default one. No reason the same
>> couldn't be applied here.
> If the original command has been replaced by a default coroutine, how
> do you construct other coroutines based on it?
By the same magic that makes everything happen. ;)
My thinking at the time put the coroutine structure as an extension of
a proc, in which case such information would have been available.
That aside...
This whole idea hinges on the displaced proc being held somewhere in
the coroutine structure for this to work at all, as it needs to be put
back when the coroutine finishes. Therefore the capacity exists for
[coroutine] to pluck it back out and construct a fresh coroutine under
another name.
>>> Another difficulty is that the original proc may take different
>>> arguments than the resume command.
>> Actually that came out rather cleanly in the wash. The original
>> proc takes initialisation arguments, while the resumed coroutine
>> gets any arguments via the return value from [yield]. Unless it
>> takes specific action, those arguments will be ignored in the
>> resuming case anyhow.
> Two cases:
> 1. resume only takes a single argument, whereas the original proc
> takes 1, in which case anybody trying to call the original will get an
> error.
Unless there is some seriously useful functionality the requires the
coroutine taking arguments (and even I can't think of anything remotely
sane there ;) ), this would be a major PITA. It utterly blocks some
functionality, while saving you from only very minor inconvenience at
best.
> 2. resume takes multiple arguments and the coroutine expects >1,
> whereas the original proc takes only 1 argument, will again result in
> an error for callers of the original proc.
Well, the thing is that the coroutine doesn't take _any_ arguments.
This got me at first, too, in the implementation being offered up. The
coroutine isn't a proc, or a command, it's an [exec] floating in a
bucket-load of magic. It's only connection back to the caller, the
only "arguments" that it'll ever see, is what pops out of [yield].
Making that a list of whatever was given to the coroutine command,
always, allows the coroutine to pretend to be a regular proc, ignore
the arguments totally, or anything in between. [lassign] can pluck out
passed words with ease, regular validation techniques can be applied to
the returned list beforehand. Whatever...
The decision seems to be between being able to make the coroutine behave
the way you want, vs. being locked into a small box that needs pointless
wrapping to escape out of.
*IF* there is some functionality that the coroutine could do with, it's
almost certainly not going to be the regular case, and can quite
readily be retro-fitted with a new command. Perhaps [coroutine] needs
to be implemented as an ensemble where the current functionality is
moved to [coroutine create], so we can have [coroutine info] and
friends, and potentially a future [coroutine invoke] or something to
obtain the functionality that will be provided by those mythical future
useful extra arguments.
> The point is that the original proc and the coroutine resume command
> can have wildly different signatures and expectations. Dynamically
> replacing one with the other is likely to create incompatibilities in
> most cases. What does it achieve?
If you're going to dynamically replace one with the other, then
presumably you already know what arguments you want. That's an utterly
trivial argument. In must such instances it just doesn't make sense to
change the signature, so don't. It really is that simple.
I just personally think it's too handy a case to ignore off-hand; a
function that is able to initialise itself, and then preserve state
between invocations without having to worry about clashes in global
variable naming, namespace pollution, etc. I've got a bunch of
procs that basically look like this;
proc some-proc {...} {
upvar #0 some-butt-ugly-long-global-variable-name myState
if { ! [info exists myState] } {
... initialise internal state ...
} else {
lassign $myState ...
}
... validate and handle new arguments ...
... do regular iteration stuff ...
set retVal ...
set myState [list ...]
return $retVal
}
In contrast, a self-replacing coroutine needs only do;
proc coro-proc {...} {
... initialise internal state ...
coroutine coro-proc apply {{...locals...} {
while { 1 } {
... do regular iteration stuff ...
yield $retVal
... validate and handle new arguments ...
}
}} ... those pre-computed locals ...
}
Although it's certainly not a show-stopper... The alternative might be
something like;
proc wrap-proc {...} {
if { [info command wrap-coro] eq "" } {
... initialise internal state ...
coroutine wrap-coro apply {{... locals ...} {
while { 1 } {
... do regular iteration stuff ...
yield $retVal
... validate and handle new arguments ...
}
}} ... those pre-computed locals ...
}
return [wrap-coro ...]
}
It's just not quite as neat, though I'll grant this particular issue is
a fairly minor one.
>>> Therefore, if the resume command could take multiple arguments then
>>> it would be impossible to distinguish between multiple separate
>>> arguments and a list. You could maybe extend yield, something like:
>> No. You simply ALWAYS return a list, ala [proc]s special "args"
>> argument. It might be slightly inconvenient, but it's an awful lot
>> more convenient than not being able to do it at all. There are
>> other possibilities also, such as allowing [yield] to take an
>> arguments list after the return value. A bit odd, but it's an idea
>> if you're desperate. ;)
> The alternative is to leave things as they are and pass a list when
> you want to pass multiple values. That has the benefit of symmetry.
Symmetry with what, exactly?!? As far as I can see, it's simply a
pointless restriction. I totally understand why it was done that way
originally, but it's just not sane to leave it that way. There either
needs to be other arguments that do something, or allow multiple
arguments to be passed through like every other command in the history
of TCL. Like I said, what on earth is it supposed to be maintaining
symmetry with?!? How many people would appreciate [proc] being
arbitrarily restricted to only accepting a single argument? I really
don't see the difference.
Heck, talking about the present implementation you're so eager to
defend, is that single argument to the coroutine optional? (I haven't
noticed it actually stated in the thread one way or the other.)
If the coroutine argument is mandatory, then that's going to be
annoying in any case were you don't need arguments, which I suspect is
probably quite a large slice of the use cases. People will be forced
to pass in useless junk arguments which don't make any sense and just
confuse things.
If the coroutine argument is optional, then how does your coroutine
tell whether an argument was actually supplied or not? Unless it's
being passed through as a list already, or [yield] is taking a
confusing probably optional pass-by-name side-channel variable, or
perhaps returning a [maybe], then it's just plain broken. And if it
returns a list so that you know whether an argument was being passed
through or not, then why the heck not accept multiple arguments?!?
>>> Coroutines should be possible to use as an implementation detail,
>>> as in my worked example converting a simple synchronous program to
>>> use the event loop: http://wiki.tcl.tk/21532
>> It gets a little more complicated when you're not writing the entire
>> application from scratch. When you're trying to retro-fit new ideas
>> into old code (which I tend to do a lot of), things get a little
>> more interesting.
> Did you read the page? The entire point of the example is that you
> can rewrite the implementation without touching the main logic. I.e.,
> using coroutines makes it *easier* to "retrofit new ideas into old
> code".
Unfortunately no, not yet. I'm having some problems with my browser
the last few days. Happened once before, too, just trying to figure out
how I fixed it last time. Not that I can imagine it making a
difference to the issue at hand. A command that only accepts one
argument is going to have a hard time replacing one that requires two
or more, without an extra layer of wrapping. Whether it's a coroutine
or not is irrelevant.
--
Fredderic
November, n.:
The eleventh twelfth of a weariness.
-- Ambrose Bierce, "The Devil's Dictionary"
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 63 days, 5:52)
miguel
> Heck, talking about the present implementation you're so eager to
> defend, is that single argument to the coroutine optional? (I haven't
> noticed it actually stated in the thread one way or the other.)
>
> If the coroutine argument is mandatory, then that's going to be
> annoying in any case were you don't need arguments, which I suspect is
> probably quite a large slice of the use cases. People will be forced
> to pass in useless junk arguments which don't make any sense and just
> confuse things.
>
> If the coroutine argument is optional, then how does your coroutine
> tell whether an argument was actually supplied or not? Unless it's
> being passed through as a list already, or [yield] is taking a
> confusing probably optional pass-by-name side-channel variable, or
> perhaps returning a [maybe], then it's just plain broken. And if it
> returns a list so that you know whether an argument was being passed
> through or not, then why the heck not accept multiple arguments?!?
It is optional - if none is supplied, {} is assumed.
The coroutine argument (as seen from outside) is just the return value of the
[yield] command (as seen from inside). That's why it is one argument defaulting
to {}: just like [return].
Neil Madden
> On Thu, 04 Sep 2008 11:15:56 +0100,
> Neil Madden <n...@cs.nott.ac.uk> wrote:
>> Fredderic wrote:
>> Two cases:
>> 1. resume only takes a single argument, whereas the original proc
>> error.
>
> Unless there is some seriously useful functionality the requires the
> coroutine taking arguments (and even I can't think of anything remotely
> sane there ;) ), this would be a major PITA. It utterly blocks some
> functionality, while saving you from only very minor inconvenience at
> best.
What do you mean by "the coroutine taking arguments" being a "PITA"?
Creation of a coroutine takes inital arguments - fair enough - and
resuming a coroutine also takes an argument (which becomes the result of
the yield that suspended it). This is essential to allow communication
between coroutines without resorting to global vars.
>> 2. resume takes multiple arguments and the coroutine expects >1,
>> whereas the original proc takes only 1 argument, will again result in
>> an error for callers of the original proc.
>
> Well, the thing is that the coroutine doesn't take _any_ arguments.
> This got me at first, too, in the implementation being offered up. The
> coroutine isn't a proc, or a command, it's an [exec] floating in a
> bucket-load of magic. It's only connection back to the caller, the
> only "arguments" that it'll ever see, is what pops out of [yield].
Coroutine has nothing to do with [exec]. Under the hood it saves away an
ExecEnv structure (AIUI), but that is just the name of the particular
structure used as an execution context for Tcl code. From the Tcl script
level it is indeed a command, and it does take an optional argument.
>
> Making that a list of whatever was given to the coroutine command,
> always, allows the coroutine to pretend to be a regular proc, ignore
> the arguments totally, or anything in between. [lassign] can pluck out
> passed words with ease, regular validation techniques can be applied to
> the returned list beforehand. Whatever...
But you now have to have special code in the coroutine to handle the
fact that someone called it expecting the original command, which it has
replaced.
> *IF* there is some functionality that the coroutine could do with, it's
> almost certainly not going to be the regular case, and can quite
> readily be retro-fitted with a new command. Perhaps [coroutine] needs
> to be implemented as an ensemble where the current functionality is
> moved to [coroutine create], so we can have [coroutine info] and
> friends, and potentially a future [coroutine invoke] or something to
> obtain the functionality that will be provided by those mythical future
> useful extra arguments.
What future extra arguments? The coroutine takes a single argument right
now.
>> The point is that the original proc and the coroutine resume command
>> can have wildly different signatures and expectations. Dynamically
>> replacing one with the other is likely to create incompatibilities in
>> most cases. What does it achieve?
>
> If you're going to dynamically replace one with the other, then
> presumably you already know what arguments you want. That's an utterly
> trivial argument. In must such instances it just doesn't make sense to
> change the signature, so don't. It really is that simple.
It's not that simple - the original proc is basically a constructor for
a coroutine, whereas the coroutine resume command is an invocation of a
running coroutine. The two are very different, and I find it hard to
imagine many compelling use cases in which one would be interchangeable
with the other.
This works right now. *If* your initialisation needs no extra arguments,
then this is a fine pattern. However, more generally you would separate
the construction/initialisation from the coroutine/instance command.
[...]
>> The alternative is to leave things as they are and pass a list when
>> you want to pass multiple values. That has the benefit of symmetry.
>
> Symmetry with what, exactly?!?
Symmetry between the way the coroutine is called and the way yield
returns data -- i.e. you pass a single value to the resume command, and
a single value is returned from [yield]. Rather than passing multiple
arguments to the coroutine and getting a list (single value) back from
yield.
As far as I can see, it's simply a
> pointless restriction. I totally understand why it was done that way
> originally, but it's just not sane to leave it that way.
Tone down the hyperbole, please. There is no loss of expressiveness by
restricting the resumption command to a single argument. You can, after
all, always restore your desired behaviour:
proc resume {coro args} { $coro $args }
This could be useful in certain cases (e.g. implementing Erlang style
actors as coroutines), but in most uses I can think of a single argument
is sufficient and so I'd prefer not to have to unpack it from a list all
the time. This can of course be wrapped in a constructor:
proc actor {name args} {
interp alias {} $name {} resume [coroutine spawn {*}$args]
}
(See the wiki for [coroutine spawn] - it just generates a unique name
for the coroutine).
[...]
> Heck, talking about the present implementation you're so eager to
> defend, is that single argument to the coroutine optional? (I haven't
> noticed it actually stated in the thread one way or the other.)
I take it from this and other comments that you haven't even bothered to
try the implementation?
-- Neil
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> Heck, talking about the present implementation you're so eager to
>> defend, is that single argument to the coroutine optional? (I
>> haven't noticed it actually stated in the thread one way or the
>> other.)
> It is optional - if none is supplied, {} is assumed.
That's what I figured. And as I said, I understand why it was done
that way, and it's fine during conception. Simple, easy, safe.
> The coroutine argument (as seen from outside) is just the return
> value of the [yield] command (as seen from inside). That's why it is
> one argument defaulting to {}: just like [return].
But [return] doesn't only return a single value. It actually returns
two values, and in one case (-code error), even more. So perhaps it
wasn't such a good example.
For one, TCL has this wonderful new command called [lassign]. And
in case you didn't feel like reading my response to Neil (who I've
noticed has a tendency to follow rather than make his own path),
[yield] isn't a command that performs a task and returns a value. It
is one part of a whole new concept in flow control, and should be
treated as such.
Please, don't force everyone to construct wrapper functions to use
their coroutines (resume), and then wrapper functions to make using the
wrapper functions easier (interp alias ... resume ...), and yet more
wrapper functions to make the wrapping mechanism itself easier to use
(that actor stuff Neil was going on about). It's getting kind of
ridiculous.
It is absolutely trivial to extract arguments from a list within the
coroutine, and modern improvements in the byte compiling are making the
performance impact more and more insignificant. It's often much less
trivial, however, to wrap those arguments up in the first place, as is
evidenced by all those wrappers I just mentioned. I would much rather
use one [lindex] on the inside, than three entirely new commands on the
outside.
And as I said to Neil, you're saying that the the coroutine command
should take one argument because [yield] returns one argument. So why
can't [yield] be changed to take a formal argument list in addition to
the argument it returns? From the implementers point of view, it
requires more work. But from an API point of view, I'd say it's an
awful lot more "correct". And if it's okay to wrap the coroutine
command (in two layers no less!), why isn't it equally okay to wrap the
[yield] command?
proc yield1 {value} {
set list [yield $value]
if { [llength $list] > 1 } {
error "I'm too sexy for this command."
}
return [lindex $list 0]
}
proc yieldX {value args} {
set list [yield $value]
return [uplevel 1 [list lassign $list {*}$args]]
}
proc yieldN {value args} {
set list [yield $value]
if { [lindex $args end] eq "args" } {
set args [lrange $args 0 end-1]
uplevel 1 [list set args [list lassign $list {*}$args]]
return [llength $list]
}
}
Take your pick. (Utterly untested, of course...)
Yet again everyone seems to be so focused on explaining why the current
state of things is right, they're blind to why it isn't. You, I can
sort of understand. It's your baby, it means more work for you to
change it, etc. Heck. Post the relevant code snippet in your reply,
I'll write the changes (maybe I'll even do it half as well as you
would), and you can paste it right back into your source again. Though
I suspect it's trivial enough a change that it'd be less of a PITA to
you for you to just do it yourself.
And finally, if you don't, how long after it goes mainstream do you
think it'll be before people start asking for the coroutine command to
take multiple arguments, anyhow? Once it looses its shine, people are
going to start looking for ways to improve it, and I'm pretty darned
sure this'll be one of the first things they'll come asking for. Then
it'll be another round of TIPping, compatibility issues, so on and so
forth.
Unless, someone can tell me what extra arguments the coroutine command
might sanely have, that wouldn't be better placed in, for example, a
separate [coroutine configure] type command. And even if there does
turn out to be something, it'd be trivial to wrap that. It's certainly
not going to be the major use case!
--
Fredderic
Introducing Microsoft's New OS: VISTA
Viruses, Infections, Spyware, Trojans, Adware
Everything you never wanted in an OS and more...!
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 64 days, 8:18)
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
[[ about single vs. multiple arguments to coroutine commands ]]
> Fredderic wrote:
>> On Thu, 04 Sep 2008 11:15:56 +0100,
>> Neil Madden <n...@cs.nott.ac.uk> wrote:
>>> Fredderic wrote:
>>> 1. resume only takes a single argument, whereas the original proc
>>> takes [more than] 1, in which case anybody trying to call the
>>> original will get an error.
>> Unless there is some seriously useful functionality the requires the
>> coroutine taking arguments (and even I can't think of anything
>> remotely sane there ;) ), this would be a major PITA. It utterly
>> blocks some functionality, while saving you from only very minor
>> inconvenience at best.
> What do you mean by "the coroutine taking arguments" being a "PITA"?
You've taken that WAY out of context...! There's two separate
statements there...
"Unless there is some seriously useful functionality the requires the
coroutine taking arguments" - By "the coroutine" I meant the coroutine
command and the underlying mechanics of the coroutine mechanism, not
the chunk of script running within the coroutine environment. If that
was the source of the confusion, then I apologise.
Now, if the coroutine command itself can do something useful with its
arguments, and I can't for the life of me imagine what that might be,
then sure. It may make sense for the pass-through argument to be a
single word, either preceded or followed by optional specially handled
arguments that the script running within the coroutine will probably
never see.
Though I think that's such an unlikely use case, and has some fairly
frightening prospects in terms of complexity on both sides of the
fence, that I'd personally rather that sort of thing be handled by a
secondary command explicitly for reconfiguring the coroutine
environment (a subject I did touch on in another post in this thread),
if and when it comes about.
That leaves the coroutine command with nothing but the arguments you're
expecting to pass through the [yield] command.
> Creation of a coroutine takes inital arguments - fair enough - and
> resuming a coroutine also takes an argument (which becomes the result
> of the yield that suspended it). This is essential to allow
> communication between coroutines without resorting to global vars.
"takes an argument" - That is the only part of that entire paragraph I
have the slightest bit of contention with. In fact, even more
specifically, just one word of it, and the wording of that has never
been really justified that I can tell. Specifically the word "an". Why
"an", and not "some"? Why one, and not many? Is there any really
compelling reason (other than saving a line or two of C code) to limit
the coroutine command to taking no more than one argument?
Does limiting it to a single word help scripts use the coroutine? Is
it helpful to a script to have to collect the arguments it wishes to
pass through into a list before invoking the command, just to have the
coroutine unpack it again anyhow? Yes, it might be useful for the
script to not have to unpack a single-item list, but is that really
worth losing the ability to determine if there even was an argument
given to the coroutine command?
On the other hand, having a coroutine command that passes all its
arguments through as a list, means the coroutine script can tell
whether its argument was an empty string, or whether there was in fact
no argument given at all. Plus you don't need to fuss with building a
list from the calling side, which means it can be plugged directly into
code that expects a regular multi-argument command without having to be
wrapped. And unpacking it is as simple as an [lassign] or a few
[lindex]s which, as it turns out, is exactly the same as you'd have
to do if you were passing it anything other than exactly one argument.
(Note, if zero arguments matters, and the empty string is considered a
valid argument, then the existing implementation forces you to wrap
even single argument invocations in a list.)
As far as I can tell, it's a lose-lose situation except for a specific
minority of use cases. Unless there is a compelling need for the
pass-through arguments to be confined to a single word, which so far as
I've seen so far, there isn't, it should be handled in the style of
[proc]s $args. To date there simply hasn't been a sane argument for
leaving it limited to one single argument.
>>> 2. resume takes multiple arguments and the coroutine expects >1,
>>> whereas the original proc takes only 1 argument, will again result
>>> in an error for callers of the original proc.
>> Well, the thing is that the coroutine doesn't take _any_ arguments.
>> This got me at first, too, in the implementation being offered up.
>> The coroutine isn't a proc, or a command, it's an [exec] floating
>> in a bucket-load of magic. It's only connection back to the
>> caller, the only "arguments" that it'll ever see, is what pops out
>> of [yield].
> Coroutine has nothing to do with [exec]. Under the hood it saves away
> an ExecEnv structure (AIUI), but that is just the name of the
> particular structure used as an execution context for Tcl code. From
> the Tcl script level it is indeed a command, and it does take an
> optional argument.
Yes, people who repeatedly argue against something just for the sake of
it get frustrating, and leads one to simple slips like [exec] instead
of [eval]. So far all I've heard you saying on this topic is "because
it is", and some mutterings about how nice and easy the current
implementation makes one tiny little specific use case, with no regard
to every other use case which it also makes more complicated for no
particularly good reason.
>> Making that a list of whatever was given to the coroutine command,
>> always, allows the coroutine to pretend to be a regular proc, ignore
>> the arguments totally, or anything in between. [lassign] can pluck
>> out passed words with ease, regular validation techniques can be
>> applied to the returned list beforehand. Whatever...
> But you now have to have special code in the coroutine to handle the
> fact that someone called it expecting the original command, which it
> has replaced.
What is your fixation on the original command?!? If it fits the usage,
then it should be possible to do it.
>> *IF* there is some functionality that the coroutine could do with,
>> it's almost certainly not going to be the regular case, and can
>> quite readily be retro-fitted with a new command. Perhaps
>> [coroutine] needs to be implemented as an ensemble where the
>> current functionality is moved to [coroutine create], so we can
>> have [coroutine info] and friends, and potentially a future
>> [coroutine invoke] or something to obtain the functionality that
>> will be provided by those mythical future useful extra arguments.
> What future extra arguments? The coroutine takes a single argument
> right now.
I'm getting that bashing head on brick wall feeling again...
>> If you're going to dynamically replace one with the other, then
>> presumably you already know what arguments you want. That's an
>> utterly trivial argument. In must such instances it just doesn't
>> make sense to change the signature, so don't. It really is that
>> simple.
> It's not that simple - the original proc is basically a constructor
> for a coroutine, whereas the coroutine resume command is an
> invocation of a running coroutine. The two are very different,
So? That's nothing but an implementation detail. If you want them to
look the same, you should be able to make them look the same. It's
that simple. And if the coroutine command accepts multiple arguments,
then you can. HOW and WHY is up to the person writing the script.
Making it possible vs. not allowing it at all, is the topic at hand.
My argument here is simply that it makes very little sense to restrict
the arguments to a single word, and a great deal of sense to pass
multiple arguments through as a list.
How many serious use cases can you see where only one argument is
required, compared to the number of use cases about where multiple
arguments will be needed, or even where the absence of that single
argument needs to be detectable?
>>> The alternative is to leave things as they are and pass a list when
>>> you want to pass multiple values. That has the benefit of symmetry.
>> Symmetry with what, exactly?!?
> Symmetry between the way the coroutine is called and the way yield
> returns data -- i.e. you pass a single value to the resume command,
> and a single value is returned from [yield]. Rather than passing
> multiple arguments to the coroutine and getting a list (single value)
> back from yield.
Well that's a limited view of things. [lassign] lets us return
multiple arguments in as natural way as TCL allows. Just because TCL
doesn't have a more obvious multiple variable assignment syntax,
doesn't make it any less useful. There are plenty of commands that
return more than a single value. Some return dicts or single
key/value pairs, others return lists.
>> As far as I can see, it's simply a pointless restriction. I totally
>> understand why it was done that way originally, but it's just not
>> sane to leave it that way.
> Tone down the hyperbole, please. There is no loss of expressiveness
> by restricting the resumption command to a single argument.
> You can, after all, always restore your desired behaviour:
> proc resume {coro args} { $coro $args }
Of course you can coerce it into any shape you feel like, with
sufficient leverage. But what's the point? A good many use cases are
going to have to do exactly that, and it's both tedious and pointless.
[coroutine] and [yield] are not [return], or whatever else you might
want to compare them to. [yield] is not a regular everyday command.
It is a new concept to TCL. Treat it as such.
Saying that because [yield] can only return a single argument, that the
coroutine command should only take a single argument, is nothing but a
cop-out. It's like saying that the -code argument to [return] should
be abolished because we have [error], [continue] and [break].
Pairing [yield] with [lassign] trivially wipes away that limitation.
[yield] is a transport mechanism, not a function. It doesn't perform
some action and return a value, it shifts the flow of control. Along
with that flow of control happens to travel some information.
> This could be useful in certain cases (e.g. implementing Erlang style
> actors as coroutines), but in most uses I can think of a single
> argument is sufficient and so I'd prefer not to have to unpack it
> from a list all the time.
The key is "that you can think of". And that is cause to put such a
harsh restriction on it why?
>> Heck, talking about the present implementation you're so eager to
>> defend, is that single argument to the coroutine optional? (I
>> haven't noticed it actually stated in the thread one way or the
>> other.)
> I take it from this and other comments that you haven't even bothered
> to try the implementation?
No, unfortunately not. But it's really beside the point. It's quite
obvious that you can write an entire application using commands that
only take one argument. But would you want to?
Try this on for a mind twist; instead of the coroutine command taking
a single arg, and [yield] returning a single value, why couldn't
[yield] take a formal args list following the value to pass out? From a
purely API perspective, is it any less correct? Let it deal with the
multiple arguments in a list issue. I think that would be incredibly
useful, and give you the benefit of consistent error messages. In the
present implementation, how is your coroutine going to produce an error
expressing what that one argument should be? And if it accepts an
empty value, how is it going to produce an error if there is no value.
[[ about displacing the generator proc with the generated coroutine ]]
> and I find it hard to imagine many compelling use cases in which one
> would be interchangeable with the other.
There've been a whole bunch of examples, none of which can't be done
differently. I said from the outset that it would be nice, and fits a
particular idea. I'm not arguing that it's worth moving the world for
this particular piece of basic functionality. Just wanting to discuss
the potential to see if it can be made easy enough to throw in as a
next-to-freebie.
In fact, it'd be interesting to see if there's a way it can be done
stably in the general case. Having it built in functionality makes it
stable against [rename]s, but otherwise it can be synthesised without
too much effort.
>> I just personally think it's too handy a case to ignore off-hand; a
>> function that is able to initialise itself, and then preserve state
>> between invocations without having to worry about clashes in global
>> variable naming, namespace pollution, etc.
>> proc coro-proc {...} {
>> ... initialise internal state ...
>> coroutine coro-proc apply {{...locals...} {
>> while { 1 } {
>> ... do regular iteration stuff ...
>> yield $retVal
>> ... validate and handle new arguments ...
>> }
>> }} ... those pre-computed locals ...
>> }
> This works right now.
Yes, but I believe that the coroutine replaces the original proc
totally, and therefore has to be an infinite loop. If [coroutine]
instead is able to displace without destroying the original proc, then
it allows us another level of flexibility where the proc could even run
normally most times, and only construct the coroutine to carry over
extra state when needed. That was the premise of wanting [yield] to
function without an explicit [coroutine]. It doesn't work quite so
neatly in this implementation, though, so most of this half of the
discussion is musings.
> *If* your initialisation needs no extra arguments, then this is a
> fine pattern. However, more generally you would separate the
> construction/initialisation from the coroutine/instance command.
You could. But if you're using coroutines to replace a single proc
that exhibits that kind of initialise-on-first-run behaviour, then
replacing the construction proc with the coroutine instance makes a
great deal of sense, because the signatures WILL be identical.
--
Fredderic
This message was sent without a virus, please delete some files
yourself.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 64 days, 8:49)
miguel
> On Thu, 04 Sep 2008 12:37:03 -0300, miguel <mso...@users.sf.net> wrote:
>
>> Fredderic wrote:
>>> Heck, talking about the present implementation you're so eager to defend,
>>> is that single argument to the coroutine optional? (I haven't noticed it
>>> actually stated in the thread one way or the other.)
>> It is optional - if none is supplied, {} is assumed.
>
> That's what I figured. And as I said, I understand why it was done that way,
> and it's fine during conception. Simple, easy, safe.
>
>
>> The coroutine argument (as seen from outside) is just the return value of
>> the [yield] command (as seen from inside). That's why it is one argument
>> defaulting to {}: just like [return].
>
> But [return] doesn't only return a single value. It actually returns two
> values, and in one case (-code error), even more. So perhaps it wasn't such
> a good example.
Uh? Are we both talking about Tcl? Absolutely EVERY command returns a single
value. Look at the api, Tcl_GetObjResult(). Or do try at the console to give
[return] more than one result argument. Or look at the man page at
http://tcl.tk/man/tcl8.6/TclCmd/return.htm
I think you are confusing a command's result with a lot of other things that a
command can do: set the return code, change some state, set some variables, open
a file, send an email, ...
> For one, TCL has this wonderful new command called [lassign]. And in case
> you didn't feel like reading my response to Neil (who I've noticed has a
> tendency to follow rather than make his own path), [yield] isn't a command
> that performs a task and returns a value. It is one part of a whole new
> concept in flow control, and should be treated as such.
Sorry to disagree: Tcl only has commands that "perform a task and return a
value". I do not think I want to change *that*.
> Please, don't force everyone to construct wrapper functions to use their
> coroutines (resume), and then wrapper functions to make using the wrapper
> functions easier (interp alias ... resume ...), and yet more wrapper
> functions to make the wrapping mechanism itself easier to use (that actor
> stuff Neil was going on about). It's getting kind of ridiculous.
Hmmm ... I do not think that everything should be coded in C; a library of
wrappers for some typical uses could be in tcllib, or even provided in a default
installation (similar to ::unknown, or via [package require]).
That is: the C implementation should IMO enable all (at least many) uses; which
one it "prefers" in the sense of not requiring wrapping is a decision that involves
(a) expected frequency of usage
(b) required performance for each usage
(c) simplicity of core code
(d) opinions
I'd be reluctant to have too many variants in the core, it's way better to have
the enabling mechanism in core and syntactic sugar in tcl wrappers. IMHO.
I would appreciate if you would provide concrete sample usage patterns (in the
wiki), together with the required "wrappers", so that we can properly weigh them
against other usage patterns.
Or maybe a description or prototype man page for what you want? Along the lines
of http://msofer.com:8080/wiki?name=Coroutines would be fine, I guess.
But maybe you mean by "don't force everyone to construct wrapper functions" that
there is one variant that does the right thing for every use case?
> It is absolutely trivial to extract arguments from a list within the
> coroutine, and modern improvements in the byte compiling are making the
> performance impact more and more insignificant. It's often much less
> trivial, however, to wrap those arguments up in the first place, as is
> evidenced by all those wrappers I just mentioned. I would much rather use
> one [lindex] on the inside, than three entirely new commands on the outside.
>
> And as I said to Neil, you're saying that the the coroutine command should
> take one argument because [yield] returns one argument. So why can't [yield]
> be changed to take a formal argument list in addition to the argument it
> returns? From the implementers point of view, it requires more work. But
> from an API point of view, I'd say it's an awful lot more "correct". And if
> it's okay to wrap the coroutine command (in two layers no less!), why isn't
> it equally okay to wrap the [yield] command?
Who said it's not OK? You can wrap every command in Tcl, why would this one not
be ok?
> proc yield1 {value} { set list [yield $value] if { [llength $list] > 1 } {
Oohhhh ... I'd be tempted to resume with an argument "\{" just to make you crash
> error "I'm too sexy for this command." } return [lindex $list 0] }
>
> proc yieldX {value args} { set list [yield $value] return [uplevel 1 [list
> lassign $list {*}$args]] }
Uhhh ... I'd really not wrap this one, how about the perfectly clear
lassign [yield $value] foo1 foo2 foo3
(apart from the crash if resumed with a non-list, of course)
> proc yieldN {value args} { set list [yield $value] if { [lindex $args end] eq
> "args" } { set args [lrange $args 0 end-1] uplevel 1 [list set args [list
> lassign $list {*}$args]] return [llength $list] } }
>
> Take your pick. (Utterly untested, of course...)
Please do test it - give some code we can analyze to try to understand what
you're saying, and most importantly what you want to do. What are the use cases
for these things? How come you are not complaining about (say) [eval] requiring
wrappers to do similar things?
You really got me confused when you started talking about [yield]'s formal
arguments, I thought all the while we were talking about the resume command's
formals (and how they map to [yield]'s return value).
> Yet again everyone seems to be so focused on explaining why the current state
> of things is right, they're blind to why it isn't. You, I can sort of
> understand. It's your baby, it means more work for you to change it, etc.
> Heck. Post the relevant code snippet in your reply, I'll write the changes
> (maybe I'll even do it half as well as you would), and you can paste it right
> back into your source again. Though I suspect it's trivial enough a change
> that it'd be less of a PITA to you for you to just do it yourself.
The current state of things is EXPERIMENTAL, not right or wrong. Are you doing
experiments, testing the functionality on actual use cases? That'd help both
debug the damn thing and decide on what the best api might be.
As to the "relevant code snippet": the thing is in cvs at
http://tcl.cvs.sourceforge.net/tcl/tcl/, help yourself. The relevant code is at
the end of generic/tclBasic.c
But the problem is *not* the work to implement. It is deciding what is the best
option. You seem to have a strong opinion, but your arguments are not coming
through clearly enough. If you want to convince, provide use cases.
I did not see myself as defending anything, just explaining what's in the core -
and why I took some design decisions. I can still be convinced that they're not
optimal. That is possibly the main reason why the thing is not TIPped yet.
> And finally, if you don't, how long after it goes mainstream do you think
> it'll be before people start asking for the coroutine command to take
> multiple arguments, anyhow? Once it looses its shine, people are going to
> start looking for ways to improve it, and I'm pretty darned sure this'll be
> one of the first things they'll come asking for. Then it'll be another round
> of TIPping, compatibility issues, so on and so forth.
Don't know, really. How would people be using them? Will there be many uses with
multiple arguments and where [list]-wrapping them is too clumsy? More than the
use cases where the [lindex] is a PITA? Is this in apps where indirection
through a wrapper is unacceptably slow (but direct calling is not)?
miguel
Fredderic wrote:
> On Thu, 04 Sep 2008 17:47:08 +0100, Neil Madden <n...@cs.nott.ac.uk> wrote: As
> far as I can tell, it's a lose-lose situation except for a specific minority
> of use cases.
^^^^^^^^^^^^^^^^^^^^^
> Yes, people who repeatedly argue against something just for the sake of it
> get frustrating, and leads one to simple slips like [exec] instead of [eval].
> So far all I've heard you saying on this topic is "because it is", and some
> mutterings about how nice and easy the current implementation makes one tiny
> little specific use case, with no regard to every other use case which it
^^^^^^^^^^^^^^^^^^^^
> also makes more complicated for no particularly good reason.
> How many serious use cases can you see where only one argument is required,
> compared to the number of use cases about where multiple
^^^^^^^^^^^^^^^^^^^^^^^^^
> arguments will be needed, or even where the absence of that single argument
> needs to be detectable?
> Of course you can coerce it into any shape you feel like, with sufficient
> leverage. But what's the point? A good many use cases are going to have to
^^^^^^^^^^^^^^^^^^^^^
> do exactly that, and it's both tedious and pointless.
WHERE ARE THEY???
> I'm getting that bashing head on brick wall feeling again...
Yeah, tell me about it ...
>>>> The alternative is to leave things as they are and pass a list when you
>>>> want to pass multiple values. That has the benefit of symmetry.
>>> Symmetry with what, exactly?!?
>> Symmetry between the way the coroutine is called and the way yield returns
>> data -- i.e. you pass a single value to the resume command, and a single
>> value is returned from [yield]. Rather than passing multiple arguments to
>> the coroutine and getting a list (single value) back from yield.
>
> Well that's a limited view of things. [lassign] lets us return multiple
> arguments in as natural way as TCL allows. Just because TCL doesn't have a
> more obvious multiple variable assignment syntax, doesn't make it any less
> useful. There are plenty of commands that return more than a single value.
> Some return dicts or single key/value pairs, others return lists.
They ALL return a single value. That value may be interpreted as a list or a
dict, sometimes a file handle, sometimes a command name. But it is a single
value for all Tcl commands.
>>> As far as I can see, it's simply a pointless restriction. I totally
>>> understand why it was done that way originally, but it's just not sane to
>>> leave it that way.
>> Tone down the hyperbole, please. There is no loss of expressiveness by
>> restricting the resumption command to a single argument. You can, after
>> all, always restore your desired behaviour: proc resume {coro args} { $coro
>> $args }
>
>
> [coroutine] and [yield] are not [return], or whatever else you might want to
> compare them to. [yield] is not a regular everyday command. It is a new
> concept to TCL. Treat it as such.
>
> Saying that because [yield] can only return a single argument, that the
> coroutine command should only take a single argument, is nothing but a
> cop-out. It's like saying that the -code argument to [return] should be
> abolished because we have [error], [continue] and [break].
Ahhh ... so you want the coroutine command to take options like [return]? That's
completely different: [return] really does take a single value to return, and
does no automatic list-wrapping for you. Which options would you like? Why?
> Pairing [yield] with [lassign] trivially wipes away that limitation. [yield]
> is a transport mechanism, not a function. It doesn't perform some action
> and return a value, it shifts the flow of control. Along with that flow of
> control happens to travel some information.
[yield] is a honest true-to-bog Tcl command. As such, it performs some action
and returns a value - like every other command. The only novelty is that it
returns twice (outside the coroutine when yielding, inside on resuming).
>> This could be useful in certain cases (e.g. implementing Erlang style
>> actors as coroutines), but in most uses I can think of a single argument is
>> sufficient and so I'd prefer not to have to unpack it from a list all the
>> time.
>
> The key is "that you can think of". And that is cause to put such a harsh
> restriction on it why?
You're thick:
(a) it is not a "harsh restriction", there is no loss in expressiveness, a
simple wrapper changes one to the other.
(b) because simpler code is ALWAYS good, but especially so for EXPERIMENTAL stuff
(c) because we hope that the community will provide use cases that may help
decide which is best. Real use cases.
(d) because lacking a strong argument to the contrary, symmetry is the deciding
factor: these commands behave just like the others. You are defining the return
value of a command? It's one value - and that's decided by Tcl, not by each
command separately. You'd like a facility to list-wrap things automatically to
construct it? It will be useful in some cases, a PITA in others.
The implementation would be trivial, and the runtime cost possibly negligible.
It is the desirability that is in doubt. Your tone is not really helping me to
analyze that.
>>> Heck, talking about the present implementation you're so eager to defend,
>>> is that single argument to the coroutine optional? (I haven't noticed
>>> it actually stated in the thread one way or the other.)
>> I take it from this and other comments that you haven't even bothered to
>> try the implementation?
>
> No, unfortunately not. But it's really beside the point. It's quite obvious
> that you can write an entire application using commands that only take one
> argument. But would you want to?
Maybe ... pipelines are useful. Forth words take a single implicit argument (the
whole stack), and implicitly return a single value (the whole stack). It does
have its charm. And *this* is really really beside the point.
Stop the rhetoric and childish accusations, show use cases.
Neil Madden
Fredderic wrote:
[...]
> "Unless there is some seriously useful functionality the requires the
> coroutine taking arguments" - By "the coroutine" I meant the coroutine
> command and the underlying mechanics of the coroutine mechanism, not
> the chunk of script running within the coroutine environment. If that
> was the source of the confusion, then I apologise.
I'm still slightly confused. I assume that by "coroutine command" you
mean the command used to resume an instance of a coroutine? That does
indeed take an argument -- the return value for the yield it resumes.
[...]
> Does limiting it to a single word help scripts use the coroutine? Is
> it helpful to a script to have to collect the arguments it wishes to
> pass through into a list before invoking the command, just to have the
> coroutine unpack it again anyhow? Yes, it might be useful for the
> script to not have to unpack a single-item list, but is that really
> worth losing the ability to determine if there even was an argument
> given to the coroutine command?
As I showed, automating this collection of arguments is trivial if you
want it. Here, again:
proc resume {coro args} { $coro $args }
For instance, I just wrote a toy Erlang style actors implementation
using the event loop. The message send is simply:
proc send {actor args} {
after 0 [list $actor $args]
yield
}
[...]
> My argument here is simply that it makes very little sense to restrict
> the arguments to a single word, and a great deal of sense to pass
> multiple arguments through as a list.
>
> How many serious use cases can you see where only one argument is
> required, compared to the number of use cases about where multiple
> arguments will be needed, or even where the absence of that single
> argument needs to be detectable?
On the contrary, I believe the majority of use cases will be one/zero
argument (e.g. almost all the new coroutine code on the wiki is
single-arg, as far as I can tell). IMO, *the* key motivating advantage
of introducing something like coroutines (or continuations) into Tcl is
that it allows you to take code of the form:
set x [do_something ...]
set y [do_other $x ...]
...
and rewrite do_something and do_other so that they can work
asynchronously and later resume that coroutine when the result is
available. This is single/zero arg (i.e. resume is being used as a
replacement for [return]). This is the primary use case that cannot
easily be expressed by other means in current Tcl. Another example
use-case would be something like a nondeterministic choice operator
where you want [do_something ...] to return multiple times trying
different values. Again, single return value, but multiple resumes at
the same place. Now, this latter example is not (AFAICT) expressible
using the current coroutine mechanism, as such coroutines are only
capable of expressing one-shot continuations. A useful discussion can be
had (and is being had) on whether this particular semantic limitation
represents a significant use case, and weighing up pros and cons of
alternatives (such as suspend/resume proposed by Lars). That's a much
more interesting discussion than whether coroutine resumption should
take multiple arguments.
[...]
>> I take it from this and other comments that you haven't even bothered
>> to try the implementation?
>
> No, unfortunately not.
[...]
> Try this on for a mind twist; instead of the coroutine command taking
> a single arg, and [yield] returning a single value, why couldn't
> [yield] take a formal args list following the value to pass out? From a
> purely API perspective, is it any less correct? Let it deal with the
> multiple arguments in a list issue. I think that would be incredibly
> useful, and give you the benefit of consistent error messages. In the
> present implementation, how is your coroutine going to produce an error
> expressing what that one argument should be? And if it accepts an
> empty value, how is it going to produce an error if there is no value.
I prefer that to [yield] returning a list, certainly.
-- Neil
Neil Madden
[...] Neil (who I've
> noticed has a tendency to follow rather than make his own path),
Oh my! Well that's certainly put me in my place! Nothing like an
ad-hominem attack for really spicing up those gaps between evidence
based reasoning.
-- Neil
Fredderic
miguel <mso...@users.sf.net> wrote:
> Fredderic wrote:
>> Just trying to get a handle on the problem for my own
>> understanding; I'm assuming from what you wrote that a variable is
>> essentially a named pointer to a pointer to the value. [upvar] would
>> then create a new named pointer to that same pointer to the value.
>> So the break down would happen if that stack frame went away while
>> we weren't looking.
> Something like that, yes. That is the reason why [upvar] as
> implemented will not let you create links as follows:
>
> % proc fail {} {upvar 0 foo ::foo}
> % fail
> bad variable name "::foo": upvar won't create namespace variable that
> refers to procedure variable
>
> As long as things are in the CallFrame stack the deletion order
> guarantees that things disappear sanely, that we never get a
> "dangling pointer".
*nods* All straight forward enough. Thanks.
>> Failing all that, [uplevel] shouldn't suffer from any of those
>> issues, and could even be used to fudge an inter-coroutine [upvar],
>> especially if the #level syntax can be extended to address a level
>> within another coroutine (most likely counting from its #0
>> upwards). It may even be an acceptable interim hack to add such
>> fudging into [upvar] to make it look like it works for consistency
>> sake, albeit painfully slowly.
> Oh - this is not "fudging upvar/uplevel", this is redesigning the
> whole concept of levels in Tcl.
nahh... Just bending them a little.
> BTW: currently yield does nothing but stop execution and shift some
> structures out of the way. It does not traverse the variable list to
> decide what to do with upvars or anything like that. Just save some
> data (currently 5 pointers or integers). As a matter of fact, to the
> suspended coroutine it just looks as if it invoked any old proc ...
> and was put in the freezer until it returns: the exact same data is
> saved/restored by NRE in proc-calling-proc situations.
Basically I think my suggestion essentially involves this:
1) all coroutines should be held in a list so [upvar] and friends can
walk it to find the one you're asking for.
2) when creating an inter-coroutine [upvar] you would set an unset hook
on the target variable to also destroy the local alias.
3) the local end of the [upvar] likewise needs an unset hook to release
the trace on the target variable.
Depending on the implementation, one or both unset hooks could be an
unravel hook on the stack frame itself, if that's any easier. These
unset hooks have nothing at all to do with TCL scripting side (which is
why I avoided the term "trace" here) and should not be visible there.
What to do next... If we can late-bind variables on demand, then having
the variable keep the reference information will allow it to be
re-bound if it's needed again. (Out of curiosity, how does [upvar]
work with target variables that don't exist yet? Are they created
immediately but just don't have a Tcl_Obj set, or are they created when
something writes to them?) Otherwise the script merely needs to not
assume that [upvar]d variables still exist after a [yield] (admittedly
not necessarily an easy thing, though, since the [yield] could happen
in a called proc).
> You may want to think of this as follows: coroutines are designed to
> run properly when called from the event loop, eg as in
> http://wiki.tcl.tk/21532. Anything else that can be done with them is
> pure gravy. Note that they can contain themselves a command
> dispatcher that does whatever you want based on what [yield] returns,
> dispatching via [switch], ensembles, an associated object (TclOO,
> XOTcl, iTcl, snit, ...), arrays or dicts containing command names,
> variable traces, whatever. So the possibilities are endless even if
> not exhaustively so, and still largely unexplored.
>
> Although it did take me a bit to get right (and am not sure I'm there
> yet), the current model is a relatively simple leveraging of the NRE
> infrastructure. No major rewrites were needed, no code was modified
> except in the command dispatch and bytecode execution areas.
*nods* And to that extent it's absolutely bloody brilliant as far as
I'm concerned.
Just these two gripes; the arbitrary limitation on coroutine command
arguments, and the lack of at least an [uplevel] command.
Having [uplevel] (or even [coroutine eval|uplevel] for now, ala
namespaces) will at least let you do anything you wanted [upvar] for,
and shouldn't suffer from any of the issues that [upvar] presently
faces because none of it persists after the command returns.
>> Just food for insanity... ;)
> Be my guest: little sanity left here anyway. And *I* will not be
> recoding major areas of Tcl from scratch to add more WIBNIs. Not this
> year at least.
lol Not asking you to do anything like that. Promise. These two
points, though, are things I think you'll get flack for once it all
hits mainstream and the honeymoon period wears off. (At least from
those like me brave enough to give an honest opinion.) I promise to
always at least TRY to look at any amazing new box from all 12 sides,
and shake what I can see really hard to help make sure no panels will
fall off. At present that's the best help I can give, so I intend to
give it (I've been copping flack as it is for spending so much time
banging away on my keyboard instead of paying attention to the family).
My other concern is that the coroutine command arguments thing is very
quickly going to become entrenched and unchangable, where the [coroutine
uplevel] thing can be added in later for 8.7 or whatever. That's the
reason I've persisted in the discussion with Neil on that point. He
drives me absolutely bonkers (as I'm sure I do in return ;) ), but he
did manage to shake loose a few frayed edges in my understanding of it
all.
--
Fredderic
If a packet hits a pocket on a socket on a port,
and the bus is interrupted as a very last resort,
and the address of the memory makes your floppy disk abort,
then the socket packet pocket has an error to report.
-- Source Unknown
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 64 days, 22:57)
Fredderic
miguel <mso...@users.sf.net> wrote:
First off, a small preamble; I do apologise if it turns out I've misread
you in the first half of this message, and you honestly do want to
consider all available possibilities. I readily accept that *I* can be
wrong on almost everything, have already admitted to being wrong on my
original understanding, and very strongly hope I am wrong on how I read
your attitude to this. But you've let slip things that suggest that
you DO know what I was asking for, and just haven't been willing to
accept a contrary perspective, or conversely have been so wrapped up in
your own vision that you haven't bothered to take notice.
Anyhow... As they say, on with the show...
> Fredderic wrote:
>> On Thu, 04 Sep 2008 12:37:03 -0300, miguel wrote:
>>> The coroutine argument (as seen from outside) is just the return
>>> value of the [yield] command (as seen from inside). That's why it
>>> is one argument defaulting to {}: just like [return].
>> But [return] doesn't only return a single value. It actually
>> returns two values, and in one case (-code error), even more. So
>> perhaps it wasn't such a good example.
> Uh? Are we both talking about Tcl? Absolutely EVERY command returns a
> single value. Look at the api, Tcl_GetObjResult(). Or do try at the
> console to give [return] more than one result argument. Or look at
> the man page at http://tcl.tk/man/tcl8.6/TclCmd/return.htm
>
> I think you are confusing a command's result with a lot of other
> things that a command can do: set the return code, change some state,
> set some variables, open a file, send an email, ...
No, I'm just viewing it a little more abstractly. Everything is
balance. Not too narrow, not too wide, we aim for just right.
But even still, in fact, the one piece of information that EVERY
command returns, be it a TCL script or a C function, is the return code.
Without that, the caller doesn't know whether to continue on, jump
left, or build an error trace. You could get by without the result
value at all, but the return code is mandatory. The result value on
the other hand is a totally optional but very popular convenience
feature, and in fact even has a default value (being the empty string).
But since the usual view is that [return] returns a value, rather than
a code, I count it for the sake of simplicity as returning two values
instead of one. Then there's ::errorInfo and so forth. These also are
not in the same realm as some variables, opening files, sending
emails... These are intrinsic to TCL's functioning, they're mostly
automatic, and hence part of the wider concept of a return value.
If you want to get picky over this, the return value is merely a
unique unnamed special-case variable. All those result functions could
be re-written to use ::_ as a storage slot instead of the
interp-provided result slot, and everything (with the _potential_
exception of a few shortcuts and bad assumptions) should continue to
work just fine. It might even be an interesting new feature (albeit
a little perlish). So if you want to insist that [return] only returns
a single value, then it's a code that it returns, with a convenience
side-effect of optionally stashing a value away in an agreed-apon slot
where the caller can optionally retrieve it from.
So I think it's you that's getting confused here, not me.
>> For one, TCL has this wonderful new command called [lassign]. And
>> in case you didn't feel like reading my response to Neil (who I've
>> noticed has a tendency to follow rather than make his own path),
>> [yield] isn't a command that performs a task and returns a value.
>> It is one part of a whole new concept in flow control, and should
>> be treated as such.
> Sorry to disagree: Tcl only has commands that "perform a task and
> return a value". I do not think I want to change *that*.
Well that's a frighteningly narrow view of things. In the absence
of special handling (aka [catch] and friends), exactly what value does
the [return] command return to the caller? Or [break], or [continue],
or [error]. They do not "perform a task and return a value" in the
general sense. Yes, they perform a task, and yes they return a value,
but those things happen across scope boundaries. From the view of the
caller, they do not return. From the view of the parent caller, they
didn't happen, it was the invoked command finishing with a specific
return code and maybe a few variables changed and/or other side
effects (like emails being sent).
It's all a matter of perspective. And it's yours and Neils perspective
that I'm addressing. All I'm asking is to look at the other side of
the box, because all the arguments I've heard so far have been from
the inside looking out. Well I'm on the outside looking in, and I'm
doing it on purpose (though partially in defencive response to real
life) by not getting intimate with the thing yet. Enough others are
doing that (Alexandre for example, is amply capable in that regard,
and from my dealings with him I'd trust his judgement on its technical
fitness quite highly) that I'm comfortable with that. Once these
issues are addressed, THEN I'll actually play with it myself. But
while I consider it fundamentally flawed (and in a trivial to fix
way and under-justified, no less), there's little point really. It'll
arrive on my desktop soon enough, when 8.6 comes through the repo's.
My focus in the meantime, is on focusing attention where I don't
believe enough has been put.
If I've gone and hurt your feelings, and those of Neil, then I
apologise. That wasn't my intention. But no decent work gets done
unless you're willing to accept that you're wrong. You asked for
comments and I'm giving you one. To discard it so off-handedly as you
appear to be doing, is just rude.
>> Please, don't force everyone to construct wrapper functions to use
>> their coroutines (resume), and then wrapper functions to make using
>> the wrapper functions easier (interp alias ... resume ...), and yet
>> more wrapper functions to make the wrapping mechanism itself easier
>> to use (that actor stuff Neil was going on about). It's getting
>> kind of ridiculous.
> Hmmm ... I do not think that everything should be coded in C; a
> library of wrappers for some typical uses could be in tcllib, or even
> provided in a default installation (similar to ::unknown, or via
> [package require]).
But it's not needed at all. That's my main gripe. You've got a choice;
three wrappers on the outside, or one (maybe) on the inside. Take the
one on the inside choice, please.
If you assume the presence of at least one wrapper function, no one as
far as I can see has put forth a reason why the coroutine command
should be constrained to exactly one argument.
Take the path of maximum usability, especially when it comes at
virtually no relative cost (I count three wrappers vs. one to be a
gain!).
> That is: the C implementation should IMO enable all (at least many)
> uses; which one it "prefers" in the sense of not requiring wrapping
> is a decision that involves
> (a) expected frequency of usage
I expect to be able to use the coroutine command, as a command. And
commands should leave their formal arguments to be dictated by the
purpose, not implementation/design details.
> (b) required performance for each usage
So forcing the use of unnecessary wrapping improves performance?
> (c) simplicity of core code
Where possible and appropriate, I agree. But you've listed it down
here at 3, right next to "opinions", which itself says something.
> (d) opinions
>
> I'd be reluctant to have too many variants in the core, it's way
> better to have the enabling mechanism in core and syntactic sugar in
> tcl wrappers. IMHO.
You don't need any variants in the core. Instead of passing a single
value, pass an array of values, just as the core does when invoking
commands. What you've gone and done is introduced a singularity.
You've introduced a special kind of command that takes at most one
argument and requires special handling. That is something that should
be kept out of the core at all cost! I've been told that on numerous
occasions by others like DKF himself.
The C side of [yield] can pass that array though, and the TCL side of
[yield] can trivially wrap it up in a list. Or conversely, the C side
of coroutine commands can wrap up the arguments list as a convenience.
> I would appreciate if you would provide concrete sample usage
> patterns (in the wiki), together with the required "wrappers", so
> that we can properly weigh them against other usage patterns.
Neil provided a set of wrappers already, and referred to wiki pages
that implement similar ideas as well. I provided the counter-example
in this thread, of wrappers on the [yield] command which leave the
caller free to use the coroutine command in whatever way is most
natural.
I would also hazard to suggest that the [yield] wrapping can be done
far more efficiently than the coroutine command wrapping. It's hard to
test without having been implemented, though.
> Or maybe a description or prototype man page for what you want? Along
> the lines of http://msofer.com:8080/wiki?name=Coroutines would be
> fine, I guess.
You know what I want. coroutine commands accepting multiple arguments,
and those arguments appearing as a $args-style list returned from
[yield]. The C side of it is more your domain, though to help out
there also, I'd recommend having it accept and pass an array of
Tcl_Obj's. But as I said, that's more your side than mine, I'm focused
on the portion I'll have to deal with, the TCL API.
If you were at all listening, I don't see how you could have missed
that. I've argued it to a level of detail greater than you've argued
the counter issue. Yes, the present form has a small amount of inertia
behind it, and the more time is spent futzing around like that the more
inertia builds up and the more resistance there is to fixing it. Which
I rather suspect is exactly what you want, even if not intentionally.
But to start throwing that out at this late stage of the discussion, and
to persist when you already know what it is I'm looking for and we've
already discussed just about every implication worth discussing
(more so with Neil, but you've been following that discussion), suggests
that your motives are even less justified than your "its my party and
I'll cry if I want to" stance. I'm sorry, but otherwise you'd be
pushing to examine the possibility yourself, or holding it back while
you elicit help to explore it if you didn't consider yourself
sufficiently resourced to do so.
I saw DKF doing that with his OO, and it made me personally feel an
awful lot better about how it would turn out, and I'm pretty sure I
saw him change direction on a few minor points here and there due to
the opinions of others. On the other hand, he didn't so much with
[dict], and there have been a few people objecting to its now confusing
locked-in sub-command naming (there's undoubtedly a few more also who
simply accept it as inevitable now, because they too kept quiet back
when it COULD have been changed).
Instead you're fighting to hold onto what you've got. It just
doesn't make sense in what is supposed to be a collaborative
environment. Your name will be forever intermingled with it regardless
of whether you accept ideas from others or not (possibly even more so).
> But maybe you mean by "don't force everyone to construct wrapper
> functions" that there is one variant that does the right thing for
> every use case?
Yes, there is. Allow every use case with minimal fuss, and keep it
consistent with existing practise. Which is that commands can take
more than one argument.
>> And as I said to Neil, you're saying that the the coroutine command
>> should take one argument because [yield] returns one argument. So
>> why can't [yield] be changed to take a formal argument list in
>> addition to the argument it returns? From the implementers point
>> of view, it requires more work. But from an API point of view, I'd
>> say it's an awful lot more "correct". And if it's okay to wrap the
>> coroutine command (in two layers no less!), why isn't it equally
>> okay to wrap the [yield] command?
> Who said it's not OK? You can wrap every command in Tcl, why would
> this one not be ok?
Just the quantity of discussion on wrapping [yield], vs. wrapping the
coroutine command.
>> proc yield1 {value} { set list [yield $value] if { [llength $list]
>> > 1 } {
> Oohhhh ... I'd be tempted to resume with an argument "\{" just to
> make you crash
Now that's a rather peevish response, and wouldn't work in any case.
The coroutine command, since it essentially wraps its arguments in a
list ala [proc]s $args, would handle that just fine.
This is what I'm talking about. Your so wrapped up in holding onto
what you've got you haven't even taking the time to properly consider
what I posted. What kind of attitude is that, from a respected
contributor to such a wide-ranging project. If I were looking for
incorporating TCL into an embedded system, hearing you respond like
that would quite frankly give me the willies.
It's time to hop down off your pedestal and get honest. Yes, you've
done a jolly good job. I'll sing your praises for that just as high
as I possibly can from down here on Earth. But it has two glaring
omissions, and I'll do whatever I possibly can to see that they don't
become a blemish on your otherwise fantastic and amply valued effort.
> > error "I'm too sexy for this command." } return [lindex $list 0] }
> > proc yieldX {value args} { set list [yield $value] return [uplevel
> > 1 [list lassign $list {*}$args]] }
> Uhhh ... I'd really not wrap this one, how about the perfectly clear
> lassign [yield $value] foo1 foo2 foo3
That's exactly what I was arguing for right at the beginning, but Neil
didn't like it, so I thought I'd throw that one in for completeness.
> (apart from the crash if resumed with a non-list, of course)
The problem with being the designer of something, is that you tend to
get lost in your own vision. Open source software can be a rewarding
and often rather humbling experience at times, but only if you allow
yourself to be open to alternative ideas. Otherwise it's just as quick
a way to fall down and poke your eye out.
You totally changed my view on how the whole coroutine thing should
work. I don't know if you have a clue about that, and quite frankly it
doesn't matter in any case. You should be proud though, you have a
convert. And I'm not an easy person to convert. But this issue, just
goes against the grain. And you're lack of willingness to give a
contrary opinion due thought is showing through rather starkly.
>> proc yieldN {value args} { set list [yield $value] if { [lindex
>> $args end] eq "args" } { set args [lrange $args 0 end-1] uplevel 1
>> [list set args [list lassign $list {*}$args]] return [llength
>> $list] } }
>> Take your pick. (Utterly untested, of course...)
> Please do test it - give some code we can analyze to try to
> understand what you're saying, and most importantly what you want to
> do. What are the use cases for these things? How come you are not
> complaining about (say) [eval] requiring wrappers to do similar
> things?
I can't test it usefully, because the underlying mechanism doesn't work
that way. The closest I could do with the current implementation, is
test my own wrappers.
Put together a variant that will function as though the coroutine is
passing an array of arguments, then I'll jump to test it. I'll gladly
test both versions if I can. And if it works acceptably, then switch
the implementations internally. Make the current one a wrapper that
passes a list containing the one item, and likewise unpack it again on
the other end. See how that goes (accepting, of course, the body of
existing code which has already been tailored to work around the
limitation).
> You really got me confused when you started talking about [yield]'s
> formal arguments, I thought all the while we were talking about the
> resume command's formals (and how they map to [yield]'s return value).
I was. What that was about, was trying to break the box you and Neil
seem to be stuck in by suggesting a fairly wild alternative. Imagine
this; instead of
set args [lassign [yield $value] fu bar]
simply writing
yield $value fu bar args
where [yield] is seen not as a command that does something and returns
a value, but as the introduction of a pause in flow and the
introduction of a new phase of the process. As a return value [yield]
could either return the number of arguments passed to allow for some
quick and easy error checking, or you could ditch the args (being a
fixed name which may already be used) and stick with the [lassign] way.
Ultimately, my personal preference would be for a whole new unrelated
command similar to [lassign] that parses formal [proc] argument
specifications and generates the expected error messages. That
could quite easily be a proc in tcllib, would be a perfect companion to
a [yield] that returns the coroutine commands arguments as a list, and
would make it easy for a regular [proc] to use args as its sole
argument, and then use it both ways as needed. I've got a few cases
where I need to recompose the arguments of a proc back into a list to
hand off to something else. Since the code for that already exists,
this is my preferred result as a whole, and what I would be pushing for
next so long as the coroutine command allows for it.
However, none of that is readily possible when everyone's designing
their code to avoid having to wrap your present implementation.
>> Yet again everyone seems to be so focused on explaining why the
>> current state of things is right, they're blind to why it isn't.
>> You, I can sort of understand. It's your baby, it means more work
>> for you to change it, etc. Heck. Post the relevant code snippet in
>> your reply, I'll write the changes (maybe I'll even do it half as
>> well as you would), and you can paste it right back into your
>> source again. Though I suspect it's trivial enough a change that
>> it'd be less of a PITA to you for you to just do it yourself.
> The current state of things is EXPERIMENTAL, not right or wrong. Are
> you doing experiments, testing the functionality on actual use cases?
> That'd help both debug the damn thing and decide on what the best api
> might be.
Not in its present implementation, no. As it is I've spent vastly much
too much time trying to argue my position and have to spend more time
being nice to real life. All the testing I can do, appears to have
already been done. I don't have the time to step through it with a
debugger, which would be the next step. People have written wrappers to
coerce it into accepting multiple arguments (which proves the point
that it's needed functionality), and I've presented wrappers which
would theoretically unwrap such multiple arguments, along with an
example exactly like your own showing how unnecessary such wrapping is
at all.
Given an API that takes multiple arguments, I'll not only test it, but
start using it. I've got a few applications that have been fudging
coroutine behaviour already, by way of wrapping [proc] and embedding
the body block(s) in a [switch] with some magic on the side. What I
don't have, is a single real use case where only one argument will do
without yet another layer of wrapping.
> As to the "relevant code snippet": the thing is in cvs at
> http://tcl.cvs.sourceforge.net/tcl/tcl/, help yourself. The relevant
> code is at the end of generic/tclBasic.c
>
> But the problem is *not* the work to implement. It is deciding what
> is the best option. You seem to have a strong opinion, but your
> arguments are not coming through clearly enough. If you want to
> convince, provide use cases.
I thought I already had, so many times that I'm starting to feel dizzy.
Trivial examples already exist a plenty. The mere fact that already
wrappers exist to cram multiple arguments through a single argument
interface IS the use case I'm talking about. How long did those
wrappers take to be brought up? As I understand it, they'd been devised
before I even got on the subject. I'm sure I could dream up yet another
trivial and pointless example, but most of the real use cases don't yet
exist, because coroutines are so unlike what's so far been available
that there simply isn't a great significant body of code waiting for it.
If you want a _real_ use case, look at the TCL single-level [proc] hacks
that are floating around the wiki. They're all more like a state
machine than a yield/resume, but try to find one that only allows a
single argument in the resume. I don't remember seeing one.
For another use case, consider that every single argument I've heard
for keeping it as it is, I've countered without a whole lot of effort,
and my counters only involve wrapping as a conceptual side issue, your
present design has raised wrappers as a necessity. If you have to wrap
something for reasons other than mere convenience, then it's not a
complete API. Sometimes that's acceptable, when the complete API is
prohibitively complex and the corresponding wrappers relatively
simple. But I don't believe that the case here, I'm just not
sufficiently intimately familiar with the C-level aspects of it to
prove that. You, on the other hand, I believe could do it almost
trivially and give us something real to test with.
Consider also that you needed to introduce a default value in the
definition of the coroutine command. That is another sign of a flawed
API, as it makes it impossible to distinguish no arguments from an
empty argument. That came out bright and true in the [maybe]
discussions, and is not an acceptable ONLY option. BETTER in terms of
API design, though more annoying, would be to insist on exactly one
argument ALWAYS.
Now, as stated already, I'm talking from the perspective of the TCL
script API. On the C side, maybe a single Tcl_Obj or even a void*
makes sense. But from TCL, having a [resume] wrapper as standard fare
(and from what I've already seen it will be standard fare, even if it's
not provided to you in core or tcllib) introduces two possibilities;
1) the coroutine will be invoked with [resume], and arguments need to
be unwrapped before use. Failure to do so will cause an error.
Passing a single argument is SOMETIMES more efficnet, but not capable of
naturally carrying multiple arguments.
2) the coroutine was invoked directly with a single argument, and it
must be treated as a value and not a list. Failure to do so will cause
an error. Passing multiple arguments may in some freakish twist of
reality be less common and hence an unnecessary slight overhead, but it
is natively capable of safely transporting a single argument.
My recommendation is to make #2 the standard. The value will ALWAYS be
wrapped; if you only want one argument, then only extract one
argument. It naturally encompasses the single-argument case with less
overhead than trying to do the opposite, it protects you from a
mismatch if passing a single argument and assuming it's a list of
arguments, and there is already plenty of established practise. Just
look at how common those foreach ... break hacks were before [lassign]
came about. People are used to breaking out a list into its
arguments. It's natural, normal practice. A single-argument API makes
it a performance consideration, and one more source of bugs.
Now, if that's not a use case, I don't know what is.
> I did not see myself as defending anything, just explaining what's in
> the core - and why I took some design decisions. I can still be
> convinced that they're not optimal. That is possibly the main reason
> why the thing is not TIPped yet.
Well, if that's the case then I apologise. Perhaps talking to Neil has
irritated me more than I realised. He does that to me for some reason.
>> And finally, if you don't, how long after it goes mainstream do you
>> think it'll be before people start asking for the coroutine command
>> to take multiple arguments, anyhow? Once it looses its shine,
>> people are going to start looking for ways to improve it, and I'm
>> pretty darned sure this'll be one of the first things they'll come
>> asking for. Then it'll be another round of TIPping, compatibility
>> issues, so on and so forth.
> Don't know, really. How would people be using them? Will there be
> many uses with multiple arguments and where [list]-wrapping them is
> too clumsy? More than the use cases where the [lindex] is a PITA? Is
> this in apps where indirection through a wrapper is unacceptably slow
> (but direct calling is not)?
My take is this;
if you're going to err, do it on the side of caution and safety; assume
that the use cases are evenly split, because there's just not enough
information to make a case for either side on that basis alone and at
present people are willing to cram their idea to fit it, if it means
they get to use the shiney new toy (something which I suspect we can
all put our hand up to being guilty of).
Nothing is ever fast enough, particularly in a scripted language, so
if one is unnecessarily slower, then try to avoid it. The fewer
wrappers the better; it makes things simpler, cleaner, and faster.
More wrappers means more overhead, more bugs, more namespace clashes,
and so forth. If it's not necessary, don't encourage it.
>> Unless, someone can tell me what extra arguments the coroutine
>> command might sanely have, that wouldn't be better placed in, for
>> example, a separate [coroutine configure] type command. And even
>> if there does turn out to be something, it'd be trivial to wrap
>> that. It's certainly not going to be the major use case!
The C API can still go either way. You can probably pass a void* for
all it matters to the TCL scripting API, as long as both ends match,
which they will do if it's being built from TCL script to begin with.
That really is your domain, does it make sense to pass an array of
Tcl_Obj's like I was suggesting, or does it make sense to pass a single
TclList object? Doing it differently means extension authors will have
to be careful, but there's plenty of existing cases for that in TCL's
API. C simply is harder than TCL script. That's why TCL exists. In
fact, passing a single Tcl_Obj at the C level means that you can pass
in a pre-built list, rather than having to decompose a list just to
have it rebuilt at the other end. So perhaps the current C API is just
fine, with the advisory that it is convention to ALWAYS pass through a
list of arguments. But then on yet the fifth or sixth hand, can there
be anything other than a TCL script at the [yield] end? Could
coroutines be used entirely within C and not see a scrap of TCL
script? That's something that I'm not qualified to answer.
--
Fredderic
"I believe OS/2 is destined to be the most important operating system,
and possibly program, of all time. As the successor to DOS, which has
over 10,000,000 systems in use, it creates incredible opportunities
for everyone involved with PCs."
--- Bill Gates - OS/2 Programmer's Guide
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 64 days, 23:50)
Fredderic
miguel <mso...@users.sf.net> wrote:
> You're thick:
The feelings mutual. :)
> (a) it is not a "harsh restriction", there is no loss in
> expressiveness, a simple wrapper changes one to the other.
A simple wrapper that's entirely unnecessary.
> (b) because simpler code is ALWAYS good, but especially so for
> EXPERIMENTAL stuff
Experimental things either fail and die, or become practical at some
stage, and in both cases usually before you decide to make it so. And
if you allow a misfeature to take hold, it will stay a misfeature.
> (c) because we hope that the community will
> provide use cases that may help decide which is best. Real use cases.
It's hard to use something that doesn't exist. I know how I'll be
using it, and I know the API I'll have to coerce it into fitting. I
know past places I would have liked to use it, and none of them fit
this model any better. May of us have been thinking about this type of
functionality for a long time, but there simply isn't an existing body
of use cases to apply it to. Yet. Provide both API's, see which one
get the most use, and kill off the other in the process of moving it
out of EXPERIMENTAL. And as I said in the other message, and as you've
already demonstrated, the use case exists; a simple wrapper fixes it!
> (d) because lacking a strong argument to the contrary, symmetry is
> the deciding factor: these commands behave just like the others. You
> are defining the return value of a command? It's one value - and
> that's decided by Tcl, not by each command separately. You'd like a
> facility to list-wrap things automatically to construct it? It will
> be useful in some cases, a PITA in others.
But symmetry to what? I don't consider it symmetry, I consider it an
incomplete API. You're paying attention to one end and ignoring the
other. You're looking at the end of the tunnel, and shaping the start
to fit. I'm asking that you look at the start of the tunnel, and shape
the end of it to fit. The analogy works in the real world, too.
> The implementation would be trivial, and the runtime cost possibly
> negligible. It is the desirability that is in doubt. Your tone is not
> really helping me to analyze that.
I've said it so many times, in so many ways; I've given small examples
of usage, I've made reference to things that I can't exactly post,
because they won't make a whole lot of sense anyhow taken out of the
context of the larger project. Most of them are client/server related,
where messages are being passed back and forth. Yes, it can be crammed
down a thin pipe, but that means redefining the existing layered API,
or putting in yet another layer of wrapping.
Web servers will benefit from coroutines, as has already been
discussed. It allows them to produce a query, and sit back waiting for
the response, then continue on with all their state intact. Can I
provide a code sample, no. Not one that would make sense, the web
server I use is my own custom built server (though I have been
thinking of migrating it to one of the main two TCL-based ones that I
know of, but I'm not sure how well they handle AJAX queries).
>> No, unfortunately not. But it's really beside the point. It's
>> quite obvious that you can write an entire application using
>> commands that only take one argument. But would you want to?
> Maybe ... pipelines are useful. Forth words take a single implicit
> argument (the whole stack), and implicitly return a single value (the
> whole stack). It does have its charm. And *this* is really really
> beside the point.
But we're not talking about Forth and the rest. We're talking about
TCL. In TCL you _could_ write an entire application taking only single
arguments. But you wouldn't want to.
There is something to that, however. Pipelines can be built quite
readily with coroutines, and were my main interest in self-displacing
ability. Not really a requirement, just a handy nick-nack if it was
readily possible. Unfortunately it comes with its own set of problems.
> Stop the rhetoric and childish accusations, show use cases.
Well, I'm sorry but all I get back is, "go do what we can't be bothered
with". Well, I can't do it either, or I'd be helping you code the
thing, and doing it myself. So I'll explain it as best I can, with
the set of real life examples I have at my disposal. Almost all of
which have already been provided by you.
The thing that bugs me is that it's an uphill battle trying to move a
stone that doesn't want to be moved. Do it, see how it turns out, and
then decide which way to go. But the available use cases are already
there in your lap, I've been trying to get you to look at them
differently.
--
Fredderic
Before you criticize someone walk a mile in their shoes.
That way, when you criticize them you're a mile away and
you have their shoes.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 65 days, 3:58)
Alexandre Ferrieux
>
> > [miguel]
> > The implementation would be trivial, and the runtime cost possibly
> > negligible. It is the desirability that is in doubt. Your tone is not
> > really helping me to analyze that.
>
> I've said it so many times, in so many ways; I've given small examples
> of usage, I've made reference to things that I can't exactly post,
If you permit a bit of genial advice: the "so many" is the key.
Diluting a valid argument in so many words condemns it to oblivion.
So if you want to get anywhere with this topic, by all means be
concise.
And do repeat in one single message of fewer than 1,000,000 ASCII
character the "critical" pair of idioms allowing all of us to quickly
grok what's obviously so painful in your experience.
-Alex
miguel
Un grand merci!
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
> Fredderic wrote:
>> "Unless there is some seriously useful functionality the requires
>> the coroutine taking arguments" - By "the coroutine" I meant the
>> coroutine command and the underlying mechanics of the coroutine
>> mechanism, not the chunk of script running within the coroutine
>> environment. If that was the source of the confusion, then I
>> apologise.
> I'm still slightly confused. I assume that by "coroutine command" you
> mean the command used to resume an instance of a coroutine? That does
> indeed take an argument -- the return value for the yield it resumes.
Yeah. The actual command named "coroutine", I write as [coroutine].
The terminology I've been using came from either you or miguel, I can't
remember which now. Confusing as it is. And it's about that return
value that we're discussing.
*frowns* Okay. I'm willing to accept that maybe I know what I'm talking
about better than I'm communicating it. Communication never was my
forte...
>> Does limiting it to a single word help scripts use the coroutine?
>> Is it helpful to a script to have to collect the arguments it
>> wishes to pass through into a list before invoking the command,
>> just to have the coroutine unpack it again anyhow? Yes, it might
>> be useful for the script to not have to unpack a single-item list,
>> but is that really worth losing the ability to determine if there
>> even was an argument given to the coroutine command?
> As I showed, automating this collection of arguments is trivial if
> you want it. Here, again:
> proc resume {coro args} { $coro $args }
Yes, wrappers around wrappers around wrappers around bloody wrappers.
And for what, exactly? What does all this achieve?
When multiple arguments aren't needed, you get a very slight performance
improvement, and what performance loss there would be due to the extra
[lindex], is in a position where it can be bytecompiled to a bare
minimum.
When multiple arguments ARE needed, the performance hit is where it'll
be largest, over in the script. And since there's now two interfaces
to the coroutine (one being the coroutine command itself, the the
being the [resume] wrapper), if you happen to forget and pass a single
argument my accident, it'll get interpreted as a list and break.
> For instance, I just wrote a toy Erlang style actors implementation
> using the event loop. The message send is simply:
> proc send {actor args} {
> after 0 [list $actor $args]
> yield
> }
Which makes absolutely no difference to any part of this discussion.
The difference from the script side is three characters.
>> My argument here is simply that it makes very little sense to
>> restrict the arguments to a single word, and a great deal of sense
>> to pass multiple arguments through as a list.
>>
>> How many serious use cases can you see where only one argument is
>> required, compared to the number of use cases about where multiple
>> arguments will be needed, or even where the absence of that single
>> argument needs to be detectable?
> On the contrary, I believe the majority of use cases will be one/zero
> argument (e.g. almost all the new coroutine code on the wiki is
> single-arg, as far as I can tell).
Please explain, then, what makes it single/zero arg?
I'm talking about allowing ANY use case, in the tightest of
situations, with the least fat. You're talking about adapting any use
case to fit it, which is possible at the expense of adding fat.
> Another example use-case would be something like a nondeterministic
> choice operator where you want [do_something ...] to return multiple
> times trying different values. Again, single return value, but
> multiple resumes at the same place. Now, this latter example is not
> (AFAICT) expressible using the current coroutine mechanism, as such
> coroutines are only capable of expressing one-shot continuations.
Why can't that be implemented within [yield]? Give it a list of
values, and have it return them each one by one until they're all
consumed. I may be missing something, going as I am entirely by your
brief summary, but it seems pretty simple to me. Of course, it
conflicts with the other idea of [yield] taking a formal arguments
list, but I never liked that idea much anyhow.
> A useful discussion can be had (and is being had) on whether this
> particular semantic limitation represents a significant use case, and
> weighing up pros and cons of alternatives (such as suspend/resume
> proposed by Lars). That's a much more interesting discussion than
> whether coroutine resumption should take multiple arguments.
But is that the same as saying more significant? I doubt it... The
important ones often aren't as interesting. Who knows, maybe this one
isn't either, but it is the one being discussed right here and now.
And yes, so you can find examples that only take zero or one argument.
So can everyone else. Simple examples are easy, and mostly irrelevant.
And yet you've also found examples that involve extra layers of wrapping
to coerce them into only taking zero or one argument. But that doesn't
seem significant to you?
I'd still rather plan to cover as many cases as possible right off the
bat, especially when the difference in implementation is so minor.
>>> I take it from this and other comments that you haven't even
>>> bothered to try the implementation?
>> No, unfortunately not.
Snip everything but that... Anyone would think you're trying to be
subtle.
>> Try this on for a mind twist; instead of the coroutine command
>> taking a single arg, and [yield] returning a single value, why
>> couldn't [yield] take a formal args list following the value to
>> pass out? From a purely API perspective, is it any less correct?
Well...?
>> Let it deal with the multiple arguments in a list issue. I think
>> that would be incredibly useful, and give you the benefit of
>> consistent error messages. In the present implementation, how is
>> your coroutine going to produce an error expressing what that one
>> argument should be? And if it accepts an empty value, how is it
>> going to produce an error if there is no value.
> I prefer that to [yield] returning a list, certainly.
You prefer what, exactly? Are you saying you prefer [yield] to take a
formal arguments list? I agree, that would be very interesting.
--
Fredderic
Really, I'm not out to destroy Microsoft. That will just be a
completely unintentional side effect.
--- Linus Torvalds
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 65 days, 6:04)
Neil Madden
This is a very implementation-specific view of Tcl's exception handling
mechanisms. It's not true for instance in the TclJava/Jacl
implementation of Tcl, in which Java exceptions are used rather than
return codes. The [catch] command implementation then just uses Java's
own try/catch and converts the exception to an appropriate error code.
In terms of Tcl's semantics, it is indeed the case that commands do some
work and then either return a value or never return. Commands like
[error] and [return] simply never return from the POV of the caller, but
instead perform a jump back up the stack. Likewise, ::errorInfo and
::errorCode are not returned -- they are set as side-effects. Of course,
you can probably win any argument if you redefine terms to include
whatever you want them to include.
[...]
> If I've gone and hurt your feelings, and those of Neil, then I
> apologise. That wasn't my intention. But no decent work gets done
> unless you're willing to accept that you're wrong. You asked for
> comments and I'm giving you one. To discard it so off-handedly as you
> appear to be doing, is just rude.
Have you considered the possibility that it might be you that is wrong?
Otherwise, you would have provided some sort of actual evidence, such as
sample use cases that cannot be implemented with the current
implementation or that are unduly difficult. So far, the few examples
you've hinted at have been trivially achievable. This is not surprising
since you are essentially just bitching about syntax, rather than
getting to grips with what is actually important -- i.e., the
expressiveness of the semantics, and the pragmatics of the
implementation. Given that you cannot even be bothered to test the
implementation for yourself, and given your apparent preference for
fiery rhetoric and iconoclastic hyperbole over actual evidence, is there
any reason why we shouldn't treat you as a petulant teenager?
[...]
> You know what I want. coroutine commands accepting multiple arguments,
> and those arguments appearing as a $args-style list returned from
> [yield].
The essential choice then comes down to this trade-off:
1. The current situation - passing 1/0 args is convenient, multiple
args requires a wrapper.
2. Your situation - passing 2+ args is convenient, but you always need
an lassign/lindex around the yield even in 1-arg case.
This is a simple trade-off. Neither is more or less expressive than the
other, as either can trivially express the other. The question then is
which case to optimise for: the 0/1 arg case, or the 2+ args case. My
opinion, based on at least some trials, is that the 0/1-arg case is much
more common. If you want to convince otherwise then the onus is on you
to come up with a list of compelling use-cases that show the 2+ arg case
is more common. Simply writing a lot of words of mostly whining and name
calling doesn't constitute an argument.
Here is your preferred coroutine interface. Feel free to use this or add
it to tcllib or do whatever you prefer with it. Just don't expect it to
become the core interface just because you stamp your feet and scream
"it's not fair, nobody ever listens to me!" at the top of your voice.
namespace eval coroutine {
namespace export create yield resume
namespace ensemble create
variable id 0
proc create {name cmd args} {
variable id
set coro ::coroutine::[incr id]
set res [::tcl::unsupported::coroutine \
$coro {*}$cmd {*}$args]
interp alias {} $name {} ::coroutine resume $coro
return $res
}
proc yield val { ::tcl::unsupported::yield $val }
proc resume {coro args} { $coro $args }
}
-- Neil
Neil Madden
[...]
>> As I showed, automating this collection of arguments is trivial if
>> you want it. Here, again:
>> proc resume {coro args} { $coro $args }
>
> Yes, wrappers around wrappers around wrappers around bloody wrappers.
> And for what, exactly? What does all this achieve?
It achieves exactly what you are screaming for in a single line. No
wrappers around wrappers, just one reusable proc.
[...]
>
>> Another example use-case would be something like a nondeterministic
>> choice operator where you want [do_something ...] to return multiple
>> times trying different values. Again, single return value, but
>> multiple resumes at the same place. Now, this latter example is not
>> (AFAICT) expressible using the current coroutine mechanism, as such
>> coroutines are only capable of expressing one-shot continuations.
>
> Why can't that be implemented within [yield]? Give it a list of
> values, and have it return them each one by one until they're all
> consumed. I may be missing something, going as I am entirely by your
> brief summary, but it seems pretty simple to me. Of course, it
> conflicts with the other idea of [yield] taking a formal arguments
> list, but I never liked that idea much anyhow.
The problem is not yielding multiple values, it is yielding multiple
values from the same place. Given code like:
set x [choose 1 2 3 4]
set y [choose 3 9 7]
if {$x**2 != $y} { fail }
we would like this code to eventually succeed with x=3, y=9. Now it is
fairly easy to make [fail] resume one or the other coroutine to pick a
next value. However, what then happens is that [fail] will return with
the new value and you don't know whether it is meant to be x or y. There
is no way using coroutines to get the code to "jump" back to the
relevant [choose] statement and then continue from there. One-shot
continuations have the same limitation. This is a well documented
limitation in the literature.
-- Neil
miguel
- A - Experimental coroutines are in HEAD and 8.6a2
* they do allow new and simpler ways to express things
* what's in there is experimental infrastructure
* please play with code and concept, help us make it better
- B - Some design decisions I consider necessary, where the alternatives I saw
fall in one or more of the following categories:
(a) semantics not clear (to me and/or generally)
(b) no hope in hell to have it in 8.6 (compat)
(c) no hope in hell to have it in 8.6 (time)
Among these:
* the coroutine bodies run in [uplevel #0]
* coroutines are not serializable
* coroutines are like one-shot continuations, not restartable
- C - Some design decisions where made with possibly insufficient info:
(1) coroutines are self-cleaning
(2) [coroutine] requires a command name ([cmd] in the rest of this post]
(3) [cmd] takes a single argument: [yield]'s inside return value
(4) [yield] takes a single argument: the outside return value
(5) [yield] always return an "ok" code, both inside and outside
Of these, only (5) restricts functionality; it was taken because *I* can
not see how to make the alternative (a) useful, (b) semantically clear, (c) play
nicely with (1). I may be wrong, eager to learn.
(1) I think is valuable: when the coroutine is exhausted there's nothing
else that can be done with it, so let the command disappear instead of burdening
the caller to clean up.
The other three are ... irrelevant at this stage! Simple script wrappers
can provide alternative syntax at minimal perf cost. They do not constrain
experimentation in any way. Should we want more than one available, standard
wrappers could be in tcllib or TIPped and "builtin". Should the perf penalty be
too large, they can at any time be coded in C (or even bytecompiled!) as solving
perf bugs.
- D - I am eager to get help making this better
(a) Looking mostly for "mental bugs", things that are stopping me from
writing a TIP right away:
* does it make sense? are there inconsistencies and semantic snafus?
* are there B-type restrictions that could be lifted in time for 8.6?
* is there more C-type stuff that I am missing, and that is losing
generality?
(b) Looking for arguments that would tip C-decisions one way or the other;
this includes primarily real use cases. Mainly interested in (1) and (5), and
possibly other decisions that I may have made unconsciously (help me find them!).
What does not make ANY sense whatsoever (to me) is long tirades about
C2-4, accusations of rigidity, "I will not use it until you change this, and
nobody else will", attempts to make some alternative look decisive by rhetoric
instead of examples. Or reading many more of those posts.
BTW, I need concrete examples of how they could be used. Not only "one could
write a proc that replaces itself", but most importantly "a proc that replaces
itself is useful for ???, and is currently impossible because ???"
- Z - this thread has turned into a WOMBAT
miguel
- C -
(0a) [coroutine] creates a command and not a special object with subcommands
Justification:
(a) provides all that is needed
(b) does not require new (sub)commands to resume, delete, ...
(c) leverages Tcl's command management code (for instance, automatic
cleanup on namespace deletion)
(d) implementation simplicity
(0b) [coroutine] runs the body up to the first [yield] instead of just
creating the command
Justification:
(a) this is not really restrictive
(b) it does not create commands that take different arguments at first and
successive calls
(c) implementation simplicity
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
> The problem is not yielding multiple values, it is yielding multiple
> values from the same place. Given code like:
>
> set x [choose 1 2 3 4]
> set y [choose 3 9 7]
> if {$x**2 != $y} { fail }
Ahhh..... Now I see. Makes sense. Why didn't you just say so in the
first place? ;)
--
Fredderic
I are Pentium of Borg. Division is futile. You will be approximated.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 65 days, 10:38)
Alan Anderson
Fredderic <my-nam...@excite.com> wrote:
> If you want to get picky over this, the return value is merely a
> unique unnamed special-case variable. All those result functions could
> be re-written to use ::_ as a storage slot instead of the
> interp-provided result slot, and everything (with the _potential_
> exception of a few shortcuts and bad assumptions) should continue to
> work just fine. It might even be an interesting new feature (albeit
> a little perlish). So if you want to insist that [return] only returns
> a single value, then it's a code that it returns, with a convenience
> side-effect of optionally stashing a value away in an agreed-apon slot
> where the caller can optionally retrieve it from.
>
> So I think it's you that's getting confused here, not me.
I've been following the discussion with varying amounts of
comprehension, understanding, and agreement on one side or the other.
But this passage has left me totally befuddled.
I apologize for my closed-mindedness, but what you're arguing goes so
strongly against my understanding of Tcl that I am actually going to
ignore it and pretend the thread ended before I read it.
Fredderic
Alexandre Ferrieux <alexandre...@gmail.com> wrote:
> On Sep 6, 11:06 am, Fredderic <my-name-h...@excite.com> wrote:
> >
> > > [miguel]
> > > The implementation would be trivial, and the runtime cost possibly
> > > negligible. It is the desirability that is in doubt. Your tone is
> > > not really helping me to analyze that.
> >
> > [freddie]
> > I've said it so many times, in so many ways; I've given small
> > examples of usage, I've made reference to things that I can't
> > exactly post,
>
> If you permit a bit of genial advice: the "so many" is the key.
> Diluting a valid argument in so many words condemns it to oblivion.
> So if you want to get anywhere with this topic, by all means be
> concise.
Tried that, didn't work either. Go figure.
You and I, we could do it. Throw ideas back and forth, a little banter
to keep the cogs moving. Didn't go anywhere in the end, but did
generate some interesting thoughts, and did set you on a path that
seems to have taken you to some interesting places. I respect a free
mind, nothing has to come from it, but the ability to dig in and follow
an idea, even one that goes against the grain, matters to me.
Over on core they're interested in what is, more than what could be.
Here, you get a lot more talk about the could be's, and of course the
cant's, but we try to fix those.
And at the end of the day, it still takes more than one person to have
an argument. (Well, it doesn't really, but then it's called something
else entirely.)
miguel, you finally posted a fantastic summary. Covers everything I
wanted to talk about. I presume it's a dump from a wiki page? Should
be, if it's not. (I _did_ mention a day or two ago that my browsers
down and out for the count... I really must get that fixed this
weekend.)
> (C-0a) [coroutine] creates a command and not a special object with
> subcommands
Did someone actually suggest that? [coroutine] itself should probably
be an ensemble. Having it "create a special object with subcommands",
though, is just plain nasty.
> What does not make ANY sense whatsoever (to me) is long tirades about
> C2-4,
Now who's having a tirade. Besides, those "tirade"s as you call them,
I was mostly calling brainstorming. It may have been a little
one-sided, but it was interesting right up until the answers started
being simplistic and repetitive. Then there was nothing but noise
until the finale. Oh well.
> accusations of rigidity, "I will not use it until you change
> this, and nobody else will", attempts to make some alternative look
> decisive by rhetoric instead of examples. Or reading many more of
> those posts.
I do believe that's a misquote... Especially the "and nobody else
will" but. That's just being crabby. I said in its present state, I'll
wait and see what happens. I hate unneccesary wrappers. I really do.
They increase the interface bloat (at least in C you have a header to
separate the signal from the noise), they stifle the performance (a
proc call is a whole lot heavier than an [lindex]). They're all round
just a drag.
> BTW, I need concrete examples of how they could be used. Not only "one
> could write a proc that replaces itself", but most importantly "a proc
> that replaces itself is useful for ???, and is currently impossible
> because ???"
As for that, it was a fun idea, but I wouldn't bother trying to
implement it. It is useful, a few uses were mentioned. Neil even
thought the idea was interesting, but I'm pretty sure we all agree it's
not interesting enough to follow through.
neil, your final effort at a creative thought wasn't particularly
impressive. But you tried. I'll give you credit for that. I was
rather dissapointed, though, I'd have thought you'd done better being
the second round. What is it with you and being unwilling to step out
of someone else's shadow?
> Have you considered the possibility that it might be you
> that is wrong?
Frequently. I even went out of my way to say that a few times. You
see, now this is what turned it into a tirade. People just don't
listen anymore.
> Here is your preferred coroutine interface. Feel free to use this or
> add it to tcllib or do whatever you prefer with it. Just don't expect
> it to become the core interface just because you stamp your feet and
> scream "it's not fair, nobody ever listens to me!" at the top of your
> voice.
It was a simple idea, with a simple implementation, and existing use
cases. None of which was addressed satisfactorially in the thread until
AFTER it became a bitch-fest.
Funny that. It's not until people have had enough, that they put their
brain into gear for one final burst of effort to come up with a
topic-ender post.
I don't know about you, but personally I think the result was worth
it. I did all the work at trying to come up with interesting ideas,
while you just sat there shaking your head going "no it isn't" and "i
can't hear you". Would probably have turned out better if you'd
participated more. Still, you did finally make an effort, and that's
what counts.
You know, one last thing that I just don't understand. If my posts are
so full of meaningless rhetoric, and I'm nothing but a "petulant
teenager", then why did you keep bothering to reply to every single
one?!? As I said up top, you can't have a bitching match with yourself,
and this was a two-against-one, no less. Well, okay, I can understand
miguel wanting to defend his baby, that was your excuse last time,
too, if I remember correctly. But why were you still in it?
*shrugs* It was fun. Must be off now. Toodles.
--
Fredderic
"Nine megs for the secretaries fair
Seven megs for the hackers scarce
Five megs for the grads in smoky lairs
Three megs for system source
One disk to rule them all
One disk to bind them
One disk to hold the files
And in the darkness grind 'em"
--- LOTR revisited, source unknown
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 65 days, 8:25)
Donal K. Fellows
> (Out of curiosity, how does [upvar]
> work with target variables that don't exist yet? Are they created
> immediately but just don't have a Tcl_Obj set, or are they created when
> something writes to them?)
They're created but hold a NULL, which Tcl interprets to mean that the
variable is unset. (Yes, there are NULLs! Oooer!)
> Having [uplevel] (or even [coroutine eval|uplevel] for now, ala
> namespaces) will at least let you do anything you wanted [upvar] for,
> and shouldn't suffer from any of the issues that [upvar] presently
> faces because none of it persists after the command returns.
Did you know that [upvar] was introduced (long long ago) so that you
didn't need to do those sorts of [uplevel] tricks any more? :-)
> If a packet hits a pocket on a socket on a port,
> and the bus is interrupted as a very last resort,
> and the address of the memory makes your floppy disk abort,
> then the socket packet pocket has an error to report.
> -- Source Unknown
The evidence is that this was written by Gene Ziegler in 1994. The
original (in its full glory) is at http://www.people.cornell.edu/pages/elz1/clocktower/DrSeuss.html
Donal.
Donal K. Fellows
> Given that you cannot even be bothered to test the
> implementation for yourself, and given your apparent preference for
> fiery rhetoric and iconoclastic hyperbole over actual evidence, is there
> any reason why we shouldn't treat you as a petulant teenager?
Goodness me, but that's a little bit too impolite for c.l.t! While
Fredderic might possibly be acting in the way that you suggest, it is
infra dig. to come out and say so. It is far better to win the
argument (amongst intelligent people) by sticking to civility and
reasoned argument, leaving the absurd posturing and grand-standing to
your opponent. Rhetorical flourishes can backfire badly if the
audience thinks they're being manipulated.
(When you're arguing from facts, I agree. And I do like your wrapper
code; neat!)
Donal.
Neil Madden
> Neil Madden wrote:
>> Given that you cannot even be bothered to test the
>> implementation for yourself, and given your apparent preference for
>> fiery rhetoric and iconoclastic hyperbole over actual evidence, is there
>> any reason why we shouldn't treat you as a petulant teenager?
>
> Goodness me, but that's a little bit too impolite for c.l.t! While
> Fredderic might possibly be acting in the way that you suggest, it is
> infra dig. to come out and say so. It is far better to win the
> argument (amongst intelligent people) by sticking to civility and
> reasoned argument, leaving the absurd posturing and grand-standing to
> your opponent. Rhetorical flourishes can backfire badly if the
> audience thinks they're being manipulated.
Well, I apologise to the regular readers of c.l.t if my words seem
overly harsh.
-- Neil
Neil Madden
[...]
>> Here is your preferred coroutine interface. Feel free to use this or
>> add it to tcllib or do whatever you prefer with it. Just don't expect
>> it to become the core interface just because you stamp your feet and
>> scream "it's not fair, nobody ever listens to me!" at the top of your
>> voice.
>
> It was a simple idea, with a simple implementation, and existing use
> cases. None of which was addressed satisfactorially in the thread until
> AFTER it became a bitch-fest.
Actually that implementation is basically just the simple wrapper that
you were so scornful of.
-- Neil
Donal K. Fellows
> The problem is not yielding multiple values, it is yielding multiple
> values from the same place. Given code like:
>
> set x [choose 1 2 3 4]
> set y [choose 3 9 7]
> if {$x**2 != $y} { fail }
>
> we would like this code to eventually succeed with x=3, y=9.
It should be noted that even if this *could* work, it would amount to
a solver for NP problems: it's easy to have a continuation that can
deliver a (countably) infinite set of values, and the third line is a
classic polynomial check. As such, the cost of the above code would
actually be extremely high (assuming NP!=P and that TIP #263 hasn't
been implemented, of course...)
(I prefer things to have understandable small-step operational
semantics.)
Donal.
Neil Madden
> Neil Madden wrote:
>> The problem is not yielding multiple values, it is yielding multiple
>> values from the same place. Given code like:
>>
>> set x [choose 1 2 3 4]
>> set y [choose 3 9 7]
>> if {$x**2 != $y} { fail }
>>
>> we would like this code to eventually succeed with x=3, y=9.
> [...]
>
> It should be noted that even if this *could* work, it would amount to
> a solver for NP problems: it's easy to have a continuation that can
> deliver a (countably) infinite set of values, and the third line is a
> classic polynomial check. As such, the cost of the above code would
> actually be extremely high (assuming NP!=P and that TIP #263 hasn't
> been implemented, of course...)
Well, the (time) cost of the *above code* should be no higher than the
equivalent using explicit loops:
foreach x {1 2 3 4} {
foreach y {3 9 7} {
if {$x**2 == $y} { return [list $x $y] }
}
}
error "no solutions"
Many programming features can be put to work on NP problems, or worse.
That doesn't equate to such a feature being intrinsically expensive.
Indeed, continuations add no more power to the language, as we can
always write everything in CPS (as in the foreach loops).
You are right though, that moving to multi-shot continuations has a
price as they generally involve copying the stack (or execEnv) at each
continuation point, whereas a coroutine can reuse the same captured
environment for multiple yield/resumes.
> (I prefer things to have understandable small-step operational
> semantics.)
I can certainly agree with that.
-- Neil
Fredderic
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote:
> Fredderic wrote:
>> (Out of curiosity, how does [upvar]
>> work with target variables that don't exist yet? Are they created
>> immediately but just don't have a Tcl_Obj set, or are they created
>> when something writes to them?)
> They're created but hold a NULL, which Tcl interprets to mean that the
> variable is unset. (Yes, there are NULLs! Oooer!)
So basically what I was thinking, and although the most straight
forward and efficient method I can think of, it's probably also the
least helpful to the issue at hand... (Don't worry, won't tell a
soul...)
>> Having [uplevel] (or even [coroutine eval|uplevel] for now, ala
>> namespaces) will at least let you do anything you wanted [upvar]
>> for, and shouldn't suffer from any of the issues that [upvar]
>> presently faces because none of it persists after the command
>> returns.
> Did you know that [upvar] was introduced (long long ago) so that you
> didn't need to do those sorts of [uplevel] tricks any more? :-)
Does rather make sense. Also as above, doesn't help much with that
binding issue miguel was talking about (otherwise he'd probably
already have done it). Still, in the short term [uplevel] or something
similar would be infinitely better than nothing. [upvar/uplevel] seem
to be a fairly integral part of TCL life...
>> If a packet hits a pocket on a socket on a port,
>> and the bus is interrupted as a very last resort,
>> and the address of the memory makes your floppy disk abort,
>> then the socket packet pocket has an error to report.
>> -- Source Unknown
> The evidence is that this was written by Gene Ziegler in 1994. The
> original (in its full glory) is at
> http://www.people.cornell.edu/pages/elz1/clocktower/DrSeuss.html
Ahhh..... Legend. heh
--
Fredderic
Klingon prompt: strike any user when ready.
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 67 days, 11:51)
Fredderic
Neil Madden <n...@cs.nott.ac.uk> wrote:
Well, actually my main issue was with a certain attitude that
particularly irks me, and the argument itself became the prime example
of it. The wrapper thing I don't like, but I'll live with if it ends up
being the final interface.
Speaking of the wrapper, though, your wrapper code will leave behind
[interp alias]s with nothing there to clean them up... That's why I
don't much like it - the amount of excess cruft just to complete the
wrapper cleanly, is significantly greater than just implementing it in
the original interface to begin with. (The [yield] sub-command also,
as provided, would have been better off mapped in...)
And being experimental as it is, what's a bit of interface duplication
if it allows you to get a better idea of which interface is preferred?
Oh, and my other suggestion for making [yield] more list-friendly was
something more like this:
proc yield {val args} {
set list [::tcl::unsupported::yield $val]
if { ! [llength $args] } {return $list}
return [uplevel 1 [list lassign $list {*}$args]]
}
Not sure I'd recommend that as a sane interface, though...
WHY does [lassign] complain if it doesn't have at least one
varName...? The semantics still hold true for zero arguments!
Anyhow... Such is life.
--
Fredderic
When a copy of your floppy's getting sloppy on the disk,
and the microcode instructions cause unnecessary risk,
then you'll have to flash your memory,
and you'll want to RAM your ROM.
Quickly, turn off your computer and be sure to tell your mom.
-- Gene Ziegler, 1994
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 67 days, 14:38)
Donal K. Fellows
> Well, actually my main issue was with a certain attitude that
> particularly irks me, and the argument itself became the prime example
> of it.
So you took out your irritation on c.l.t? Fie for shame!
> The wrapper thing I don't like, but I'll live with if it ends up
> being the final interface.
>
> Speaking of the wrapper, though, your wrapper code will leave behind
> [interp alias]s with nothing there to clean them up... That's why I
> don't much like it - the amount of excess cruft just to complete the
> wrapper cleanly, is significantly greater than just implementing it in
> the original interface to begin with. (The [yield] sub-command also,
> as provided, would have been better off mapped in...)
You've singularly failed to explain what the problem is with the
interface though. :-) (I'll not comment on the problems with the
implementation for now; proofs-of-concept are often not great when
examined too closely, but little is to be learned from the exercise.)
> WHY does [lassign] complain if it doesn't have at least one
> varName...? The semantics still hold true for zero arguments!
Because TIP#323 hasn't been approved yet.
Donal.
Evil Son
<donal.k.fell...@manchester.ac.uk> wrote:
> Fredderic wrote:
> > Well, actually my main issue was with a certain attitude that
> > particularly irks me, and the argument itself became the prime example
> > of it.
>
> So you took out your irritation on c.l.t? Fie for shame!
>
> > The wrapper thing I don't like, but I'll live with if it ends up
> > being the final interface.
>
> > Speaking of the wrapper, though, your wrapper code will leave behind
> > [interp alias]s with nothing there to clean them up... That's why I
> > don't much like it - the amount of excess cruft just to complete the
> > wrapper cleanly, is significantly greater than just implementing it in
> > the original interface to begin with. (The [yield] sub-command also,
> > as provided, would have been better off mapped in...)
>
> You've singularly failed to explain what the problem is with the
> interface though. :-) (I'll not comment on the problems with the
> implementation for now; proofs-of-concept are often not great when
> examined too closely, but little is to be learned from the exercise.)
I might have this wrong (after all I've spent 6 hours in pointless
testing ...) but hasn't Fredderic said:
1. That the default interface is not sufficiently expressive for what
he wants to do: [coroutine] or whatever the command is should take a
list of arguments rather than a single word?
2. He would need an ungainly wrapper (a question of syntactic style).
3. Wrappers are over-used in Tcl and he is sick of having to write
them to get a more convenient interface.
4. He wishes we would get the interfaces right in the first place.
5. If the current implementation takes hold he'll be stuck with an
inferior interface - stuck with having to use wrappers.
I cannot comment on the merits of his case but if the current
implementation of coroutines were kept provisional and we used Neil's
wrapper to provide Fredderic's interface, Frederic and the community
will be able to bash out use cases and we'll soon see whether the
current interface is indeed sub-optimal. I don't think Miguel needs
to provide a C implementation of the alternative interface for this
kind of exploration to proceed.
(I do believe that the current interface is the "standard" one but
let's not argue about why standards cannot be improved upon.)
Ball appears to be in the community's court - lets see some more
examples/uses with both interfaces.
Regards
Evil Son
- putting the evil into testing
Donal K. Fellows
> I might have this wrong (after all I've spent 6 hours in pointless
> testing ...) but hasn't Fredderic said:
> 1. That the default interface is not sufficiently expressive for what
> he wants to do: [coroutine] or whatever the command is should take a
> list of arguments rather than a single word?
I reckon on the whole that there's a can of worms here. Surely if
there's syntactic variation, then the coroutine instance command
should allow probing of that variation (just as any other command
does)? And surely we need a way to yield an error without terminating
the coroutine? (Without that, we can't handle the checking of the
syntax in the coroutine instance, and would need a wrapper anyway, but
in that case we could have made do with a single argument...)
> 2. He would need an ungainly wrapper (a question of syntactic style).
> 3. Wrappers are over-used in Tcl and he is sick of having to write
> them to get a more convenient interface.
> 4. He wishes we would get the interfaces right in the first place.
As I indicated above, what is correct is *hard*...
> 5. If the current implementation takes hold he'll be stuck with an
> inferior interface - stuck with having to use wrappers.
But his proposals don't solve things enough. Partial solutions can
easily just make things worse. (Alas; life would be easier if we could
always use local optimization strategies for improving stuff.)
Donal.
Arnold Snarb
> Donal K. Fellows wrote:
>> Fredderic wrote (and wrote, and wrote, and wrote):
>>> [... elided ...]
>> You've singularly failed to explain what the problem is with the
>> interface though. :-)
>
> testing ...) but hasn't Fredderic said:
> 1. That the default interface is not sufficiently expressive for what
> he wants to do: [coroutine] or whatever the command is should take a
> list of arguments rather than a single word?
Yes, I think that's what he said. He's wrong.
> 2. He would need an ungainly wrapper (a question of syntactic style).
This is true. However, if Fredderic's suggestions were adopted,
then *everybody else* would need to use an ungainly wrapper instead.
Since all the early adopters and experimenters seem to
be content with the current interface; and since Fredderic
by his own admission hasn't even played with the beast yet;
and furthermore since (as DKF notes above) Fredderic has
not provided a *single* use case where multiple arguments
to a coroutine resumption would be useful other than
"Well what if somebody wants to pass multiple arguments
to a coroutine resumption?", it seems pretty clear that
making *him* use an ungainly wrapper instead of *everybody else*
is the right tradeoff.
> 3. Wrappers are over-used in Tcl and he is sick of having to write
> them to get a more convenient interface.
He's wrong.
Tcl is all about the wrappers. That's what you *do* with Tcl:
you take a facility that's too low-level, or in some cases just
plain awkward, and you wrap it up in a more convenient interface,
and then you use that interface instead. Wrapping stuff is what
Tcl is *for*.
> 4. He wishes we would get the interfaces right in the first place.
I want that too. And Miguel *did* get the interface right.
Fredderic is off on another planet.
> 5. If the current implementation takes hold he'll be stuck with an
> inferior interface - stuck with having to use wrappers.
And if his suggestions take hold, we'll all be
stuck with an ungainly foundation that will turn
into another [lsearch]. No thanks.
> I cannot comment on the merits of his case [...]
I can.
--Arnold
Neil Madden
> Evil Son wrote:
>> I might have this wrong (after all I've spent 6 hours in pointless
>> testing ...) but hasn't Fredderic said:
>> 1. That the default interface is not sufficiently expressive for what
>> he wants to do: [coroutine] or whatever the command is should take a
>> list of arguments rather than a single word?
>
> I reckon on the whole that there's a can of worms here. Surely if
> there's syntactic variation, then the coroutine instance command
> should allow probing of that variation (just as any other command
> does)? And surely we need a way to yield an error without terminating
> the coroutine? (Without that, we can't handle the checking of the
> syntax in the coroutine instance, and would need a wrapper anyway, but
> in that case we could have made do with a single argument...)
I was tackling just this problem the other day, trying to design an
event loop + coroutines Erlang-style actors implementation. I settled on
sending back a pair of options dict and message as a list (via a wrapper):
proc send {actor args} {
after 0 [list $actor [coroutine current] $args]
lassign [coroutine yield] message opts
return -options $opts $message
}
# Erlang-style receive
proc receive cases {
...
set rc [catch { upapply 1 $method {*}$args } reply options]
reply $sender $reply {*}$options
}
proc reply {actor message args} {
after 0 [list $actor [list $message $args]]
}
Works quite well in this case. It might make sense for [yield] to accept
the same options as [return], but in this case it's actually a resume
that needs to pass these options. This is another argument for keeping
yield/resume single-arg for now: it leaves open the possibility of
adding such options if deemed necessary/convenient in the future.
-- Neil
Evil Son
Precisely the sort of thing Miguel would like us to do. Hopefully
we'll see more of this kind of thing on the wiki. I do realise that
it is extra work but it would be great for the community if we get
simple use cases for the new facility in code - the simpler the
better. What would seem obvious to the likes of Neil would be a good
tutorial for me.
One good thing with Fredderic's position is that it has elicited some
information on design decisions.
Wouldn't it be great if we had something like perl6's exegeses. To wit
> Just stating what I understand and where I stand
> - A - Experimental coroutines are in HEAD and 8.6a2
> * they do allow new and simpler ways to express things
Where is Conway when you need him? (I'm not suggesting that Miguel
should have provided examples by the way - too early for the exegeses
type material)
Sorry, not quite my place, but ... if I may
[I've been pumping gas for most of the night, cut me a break :-]
fact is, good documentation is hard, harder than coding. And I think
that drh, Mark Roseman, Flynt, the Tcl tutorial team, Welch et al,
Harrison/McLennan, several wiki and comp.lang.tcl contributors and so
many unsung heroes are not only straight-out brilliant but also simply
above-the-competition generous with their knowledge.
back to my testing then ...
Evil Son
Evil Son
Evil Son
> we'll see more of this kind of thing on the wiki. I do realise that
> it is extra work but it would be great for the community if we get
> simple use cases for the new facility in code - the simpler the
> better. What would seem obvious to the likes of Neil would be a good
> tutorial for me.
Wow, there's some cool actually understandable stuff happening on the
wiki with coroutines!
miguel
> miguel, you finally posted a fantastic summary. Covers everything I
> wanted to talk about. I presume it's a dump from a wiki page? Should
> be, if it's not. (I _did_ mention a day or two ago that my browsers
> down and out for the count... I really must get that fixed this
> weekend.)
Neil Madden
>> Works quite well in this case. It might make sense for [yield] to accept
>> the same options as [return], but in this case it's actually a resume
>> that needs to pass these options. This is another argument for keeping
>> yield/resume single-arg for now: it leaves open the possibility of
>> adding such options if deemed necessary/convenient in the future.
>
> we'll see more of this kind of thing on the wiki. I do realise that
> it is extra work but it would be great for the community if we get
> simple use cases for the new facility in code - the simpler the
> better. What would seem obvious to the likes of Neil would be a good
> tutorial for me.
I've been busy developing worked examples for essentially this reason:
some will end up in the TIP as motivating use-cases and examples,
some/all will also end up in the Tcl tutorial for 8.6 when I (or other
maintainers) get round to updating it (need to check the 8.6 release
calendar...).
>
> One good thing with Fredderic's position is that it has elicited some
> information on design decisions.
>
> Wouldn't it be great if we had something like perl6's exegeses.
Well, the docs and examples will get written. There is some decent
material appearing on the wiki already. Of course, we have the advantage
(over Perl 6) of an actual implementation!
-- Neil
Fredderic
"Donal K. Fellows" <donal.k...@manchester.ac.uk> wrote:
Evil Son expressed very well what I was unable to. I do apologise to
all for the noise, it is a limitation that I face for reasons I really
don't feel like going into here and now. But since it went as far
down the tubes as it did, I will try to express once and for all what
happened, at least from my perspective, without repeating too much of
what Evil Son so aptly summarised, and a few bits he missed.
> Fredderic wrote:
>> Well, actually my main issue was with a certain attitude that
>> particularly irks me, and the argument itself became the prime
>> example of it.
> So you took out your irritation on c.l.t? Fie for shame!
Actually, no. Firstly, that's where it was, so that's where it got
said. But more relevantly, the bulk of the thread transpired very
fast. And I have to admit, I was trying to bring it back on track but
couldn't quite find the way (irritation does tend to limit ones field
of view a little) through the constant "please explain yet again
because I really can't be bothered listening, but I'm going to
counter-argue anyhow just for the sake of it". Yes, I'm sure that's
probably how I came across myself, in hindsight, but as Evil Son goes
on to point out, the information is there, and you for one apparently
didn't see it, or you wouldn't have made the next statement below.
In the meantime, since I couldn't figure out myself what I hadn't
already said, and was being greeted with "please explains" instead of
some sign of comprehension, that's what I did. I simply explained it
yet again, and again, hoping sooner or later to hit a variant that
would express the issue sufficiently, of what I'd already said
previously. Certainly the wrong technique, and I do apologise for the
noise, but I'm a little strange that way at times.
Note, however, the base topic had also evolved a little thoughout the
thread, at least from my perspective (which is the main reason I hadn't
stopped to summarise; my argument wasn't static, it was just waiting
for constructive input from the other side). It would have been much
better for everyone if the others involved would have at least tried to
follow suit and actively discuss alternatives even if they didn't agree
with them. It's all good and well for everyone to jump on the topic
now after the fact. Where were they during it?
>> The wrapper thing I don't like, but I'll live with if it ends up
>> being the final interface.
>>
>> Speaking of the wrapper, though, your wrapper code will leave behind
>> [interp alias]s with nothing there to clean them up... That's why I
>> don't much like it - the amount of excess cruft just to complete the
>> wrapper cleanly, is significantly greater than just implementing it
>> in the original interface to begin with. (The [yield] sub-command
>> also, as provided, would have been better off mapped in...)
> You've singularly failed to explain what the problem is with the
> interface though. :-) (I'll not comment on the problems with the
> implementation for now; proofs-of-concept are often not great when
> examined too closely, but little is to be learned from the exercise.)
Again, that's clearly not the case. See Evil Son's response which
expresses my issues better than I could. Yours is the attitude I was
speaking of. I explained things several ways, sometimes more concise
than others, but the response was always the same; instead of
accepting the idea as offered and seeing where it leads, you don't even
know what I was talking about.
In the end, I may well end up that I was wrong. Then so be it. At
least the points were finally expressed, and it eventually appears to
have resolved into a useful discussion. For that alone, it was worth
it.
Along the way, I also expressed a few alternative APIs that would be
made readily possible by a naturally multi-argument coroutine
(resume) command. Not that they're not possible now, it just seems
bizarre to have a mountain of wrappers muddying the waters of what
should be a very clean but flexible API (talking specifically of the
TCL side of the API). Coroutines are a fairly fundamental feature,
after all, and as you say, certainly not complete.
Also, the current state of single optional arguments is just broken
in TCL. One only needs to observe the volume of calls for NULLs, the
whole [maybe] issue, and so forth, to see that. Given a single
optional argument allowing ANY value, there's no way to conclusively
determine whether any argument was specified at all (most languages
have either NULLs or some kind of argc, so far as I can tell). At least
with [proc], you can provide a sentinel value. Here, you can't even do
that. Being able to use [llength] to determine whether there IS an
argument, makes that possible without having to mess about with
specifying formal arguments for the coroutine (resume) command.
And thus I was pushing the API of multiple arguments. In the complex
case where we need [lassign] or multiple [lindex]s, try not to add extra
cruft (ie. those wrappers which were suggested to cram multiple
arguments through a single formal argument). In the simple case where
we only need one argument, perhaps a single [yield1] wrapper that
essentially just does an [lindex 0] for convenience. It's a balance
that I was looking for, not a wrapper-heavy kludge for a simple problem.
I've already said all that several times over, though. Does it make any
more sense _this_ time?
I'll add also that the "show use cases" argument falls short for
experimental interfaces, because it tends to breed esoteric use-cases,
most of which will never see the light of day past the experimental
stage, and those that do, mostly only do so in shaded corners where few
wonder. Most of what's left, while held up as proof that one interface
is perfect, can just as easily be re-worked by those who grok it to
demonstrate the virtues of the other. Plus when one interface is
presented, people will tend to envision their use cases to fit, so
unless it's glaringly wrong, opposing use cases simply won't
materialise. It's not that the use cases don't exist, it's just that
people are more interested in doing what they want to be doing, than
doing the extra work to make it fit an interface that is just as
plausible but doesn't actually exist.
In short, given a use case that does need to pass multiple values,
people will find a way to encapsulate or avoid representing those extra
values, even at the expense of some of their original concept, just so
they can get on with the more interesting parts. Use cases are a poor
measure of an experimental interface. I'd suggest a good number of
the present interface mistakes had perfectly plausible use cases also.
Which is why I much prefer open discussion with grass roots use cases,
if any are presented at all. It's the boring mundane ones that count,
the others can be adapted to fit.
Erroring is an issue that was on the back of my mind, but I really
wanted to get the current one out of the way first. [yield] by the
looks of it, almost needs to be a [return] clone. [return]
and friends would still cause the coroutine to terminate as it
presently does.
Again, I may well be wrong on all of this. But so may the existing
interface. Most likely the optimal interface is somewhere in between
that hasn't been found yet. But regardless, the present use cases I've
seen are being written to work with existing interface, not to prove
it. And they shouldn't be used to do so. In any case, what's needed is
discussion, and that's what I tried to have.
>> WHY does [lassign] complain if it doesn't have at least one
>> varName...? The semantics still hold true for zero arguments!
> Because TIP#323 hasn't been approved yet.
Yes, and hopefully it will soon.
--
Fredderic
Debian/unstable (LC#384816) on i686 2.6.23-z2 2007 (up 72 days, 23:35)
Donal K. Fellows
> Also, the current state of single optional arguments is just broken
> in TCL. One only needs to observe the volume of calls for NULLs,
There was a loud volume, but only from some people. There were also
many calls (often quieter) for NULLs to be resisted. The reality is
that NULL represents an absence of data, and that's an unset variable.
> the whole [maybe] issue, and so forth, to see that.
The [maybe] stuff is fine, but doesn't need any core changes to shine.
> Given a single
> optional argument allowing ANY value, there's no way to conclusively
> determine whether any argument was specified at all (most languages
> have either NULLs or some kind of argc, so far as I can tell).
If you want to have full control, use 'args' and process it all
yourself. Except with the [yield] stuff, which is an experimental
interface.
> At least
> with [proc], you can provide a sentinel value. Here, you can't even do
> that. Being able to use [llength] to determine whether there IS an
> argument, makes that possible without having to mess about with
> specifying formal arguments for the coroutine (resume) command.
I'm also not that worried for now *because* of that very experimental-
ness.
> I've already said all that several times over, though. Does it make any
> more sense _this_ time?
I see the surface concerns, but not why it matters so much to you. But
that could be the jet-lag on my end.
(Try making what you write shorter. Makes it easier to digest when I'm
stuck in a conference...)
Donal.