ANN: curry.tcl - curried commands and lambdas in pure Tcl

0 views
Skip to first unread message

Alexandre Ferrieux

unread,
May 5, 1999, 3:00:00 AM5/5/99
to
Here: http://place.net/~af/tcl/curry.tcl is my first attempt at very
tiny code that allows to use 'curried commands' and lambda expressions
in pure Tcl (see description below).

I am posting it here at a very early stage for two reasons:

- to collect feedback/ideas, as usual

- to let people see by themselves to what extent it is harmless
('cause it does break scripts with space in procnames); if it
proves so, the idea is to turn it into a core patch, with an
established record of side-effects.

# Principle:

# Uses the 'unknown' default command handling mechanism to
# recursively split the first word of command names, so that names
# with spaces are interpreted as so called *curried* commands, i.e.
# incomplete commands where the remaining tokens are prepended to the
# actual arglist:
#
# % set x "puts stderr"
# % $x Hello
# Hello

# An example application is complex callable constructs which don't
# pollute the 'proc' naming space, like lambda expressions:

# % set square [lambda x {expr {$x*$x}}]
# % $square 3
# 9

# A more elaborate one is this functional lambda:

# % set compose [lambda {f g} {lambda x "$f \[$g \$x\]"}]
# % set quad [$compose $square $square]
# % $quad 3
# 81

-Alex

Wolfgang S. Kechel - Patzschke + Rasp GmbH

unread,
May 5, 1999, 3:00:00 AM5/5/99
to
Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> writes:

I'd like to have anonymous functions in Tcl, but there are two drawbacks:

1. generally, I see no difference in polluting the namespace with procedures
names (stemming from proc) and variables (set square ...).
2. the last two examples show that the price paid for it is high: The term has
to be compiled/evaluated each time it is called.

--
Wolfgang Kechel
Email: wolf...@prs.de http://www.prs.de
Patzschke + Rasp GmbH Bierstadter Str. 7 D-65189 Wiesbaden
Phone: +49-611-1731-26 FAX: +49-611-1731-31

Donal K. Fellows

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
In article <yq3e1bl...@smoira.prs.de>,
Wolfgang S. Kechel - Patzschke + Rasp GmbH <wolf...@prs.de> wrote:
> Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> writes:
[ much elided - see <URL:http://place.net/~af/tcl/curry.tcl> ]

Neat work, Alex! Next stop; closures.

> I'd like to have anonymous functions in Tcl, but there are two drawbacks:
>
> 1. generally, I see no difference in polluting the namespace with
> procedures names (stemming from proc) and variables (set square
> ...).

But you can use arrays. That makes a big difference...

> 2. the last two examples show that the price paid for it is high:
> The term has to be compiled/evaluated each time it is called.

It may be possible to improve the efficiency at some point. It isn't
always a good idea to assume that just because the performance is poor
now that it will always be so. (I can remember that lists used to be
horrendously expensive if you weren't extremely careful...)

Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ fell...@cs.man.ac.uk
-- The small advantage of not having California being part of my country would
be overweighed by having California as a heavily-armed rabid weasel on our
borders. -- David Parsons <o r c @ p e l l . p o r t l a n d . o r . u s>

Alexandre Ferrieux

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
>
> I'd like to have anonymous functions in Tcl, but there are two drawbacks:
>
> 1. generally, I see no difference in polluting the namespace with procedures
> names (stemming from proc) and variables (set square ...).

Nobody forces you to store them in variables, it was just an example.
The main use is to pass lambdas as arguments of compose them, like in
the second case !

> 2. the last two examples show that the price paid for it is high: The term has
> to be compiled/evaluated each time it is called.

Sorry, my knowledge of the core is insufficient here: do you mean that
when you eval or uplevel some source code stored in a variable (array
member in my case), the compiled bytecode is thrown away after execution
instead of being stored as the internal repr of the Tcl_Obj whose string
repr is the given source ? Ugh...

-Alex

Wolfgang S. Kechel - Patzschke + Rasp GmbH

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
fell...@cs.man.ac.uk (Donal K. Fellows) writes:

> > 2. the last two examples show that the price paid for it is high:
> > The term has to be compiled/evaluated each time it is called.
>

> It may be possible to improve the efficiency at some point. It isn't
> always a good idea to assume that just because the performance is poor
> now that it will always be so. (I can remember that lists used to be
> horrendously expensive if you weren't extremely careful...)

Right, I totally agree.
Maybe there is a friend of TclProcCompileProc that can be used, so one can
provide a command 'lambda' that returns a precompiled anonymous function?

Paul Duffin

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
>
> Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> writes:
>
> > Here: http://place.net/~af/tcl/curry.tcl is my first attempt at very
> > tiny code that allows to use 'curried commands' and lambda expressions
> > in pure Tcl (see description below).
> >
> > # An example application is complex callable constructs which don't
> > # pollute the 'proc' naming space, like lambda expressions:
> >
> > # % set square [lambda x {expr {$x*$x}}]
> > # % $square 3
> > # 9
> >
> > # A more elaborate one is this functional lambda:
> >
> > # % set compose [lambda {f g} {lambda x "$f \[$g \$x\]"}]
> > # % set quad [$compose $square $square]
> > # % $quad 3
> > # 81
>
> I'd like to have anonymous functions in Tcl, but there are two drawbacks:
>
> 1. generally, I see no difference in polluting the namespace with procedures
> names (stemming from proc) and variables (set square ...).

It is just an example of how lambda functions can be used, if you need
a named function then you should just use a named function, if you
need an unnamed function then use lambda.

> 2. the last two examples show that the price paid for it is high: The term has
> to be compiled/evaluated each time it is called.
>

If that bothers you then you need Feather which not only provides both
lambda functions and curried functions it also has vectors, maps, the
ability to easily override existing commands without worrying about
renaming, generic functions to iterate over containers / sequences.

I am busily working on building a binary installation for Windows so
people can have a play with it. (I am a Unix person at heart but
configure just takes so long).

--
Paul Duffin
DT/6000 Development Email: pdu...@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880

Paul Duffin

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Alexandre Ferrieux wrote:
>
> Here: http://place.net/~af/tcl/curry.tcl is my first attempt at very
> tiny code that allows to use 'curried commands' and lambda expressions
> in pure Tcl (see description below).
>
> I am posting it here at a very early stage for two reasons:
>
> - to collect feedback/ideas, as usual
>

Please place it in its own namespace, if you wanted to add it to the
core then make sure that it goes in ::tcl:: namespace.

> - to let people see by themselves to what extent it is harmless
> ('cause it does break scripts with space in procnames); if it
> proves so, the idea is to turn it into a core patch, with an
> established record of side-effects.
>

I have not exhaustively tested it so do not know exactly what it will
break, however it not only breaks if proc names have spaces, it also
breaks if namespaces have spaces too.

> # Principle:
>
> # Uses the 'unknown' default command handling mechanism to
> # recursively split the first word of command names, so that names
> # with spaces are interpreted as so called *curried* commands, i.e.
> # incomplete commands where the remaining tokens are prepended to the
> # actual arglist:
> #
> # % set x "puts stderr"
> # % $x Hello
> # Hello
>

> # An example application is complex callable constructs which don't
> # pollute the 'proc' naming space, like lambda expressions:
>
> # % set square [lambda x {expr {$x*$x}}]
> # % $square 3
> # 9
>
> # A more elaborate one is this functional lambda:
>
> # % set compose [lambda {f g} {lambda x "$f \[$g \$x\]"}]
> # % set quad [$compose $square $square]
> # % $quad 3
> # 81
>

> -Alex

Alexandre Ferrieux

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
>
> Right, I totally agree.
> Maybe there is a friend of TclProcCompileProc that can be used, so one can
> provide a command 'lambda' that returns a precompiled anonymous function?

As I was wondering in a nearby post, do you know if currently the
compiled bytecode of the source in $x is thrown away after each
iteration of:

set x "some code"
while {1} {eval $x}

Intuitively, the Tcl_Obj architecture should allow $x to hold an object
whose string repr is "some code" and whose internal repr is the
correponding bytecode...Ain't it the case ? Or is it outright impossible
? Paul, would it be a candidate feature for Feather, which does have a
more elaborate handling of dual representations ?

-Alex

Paul Duffin

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
>
> fell...@cs.man.ac.uk (Donal K. Fellows) writes:
>
> > > 2. the last two examples show that the price paid for it is high:
> > > The term has to be compiled/evaluated each time it is called.
> >
> > It may be possible to improve the efficiency at some point. It isn't
> > always a good idea to assume that just because the performance is poor
> > now that it will always be so. (I can remember that lists used to be
> > horrendously expensive if you weren't extremely careful...)
>
> Right, I totally agree.
> Maybe there is a friend of TclProcCompileProc that can be used, so one can
> provide a command 'lambda' that returns a precompiled anonymous function?
>

The lambda that is provided as part of Feather uses some internal
functions so that it behaves exactly like a proc with the same args
and body would, e.g. the first time through it will byte compile it,
second and subsequent times through it will run the byte code, it also
handles namespaces properly too so you can take some context with you.
e.g.
% namespace eval foo {
variable a 12

variable add12 [lambda {x} {
variable a
expr {$x + $a}
}]
}

% set a 99
99
% $foo::add12 2
14

Paul Duffin

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Donal K. Fellows wrote:
>
> In article <yq3e1bl...@smoira.prs.de>,

> Wolfgang S. Kechel - Patzschke + Rasp GmbH <wolf...@prs.de> wrote:
> > Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> writes:
> [ much elided - see <URL:http://place.net/~af/tcl/curry.tcl> ]
>
> Neat work, Alex! Next stop; closures.
>

What does a closure mean to you from a Tcl point of view ?
If I remember correctly from my lisp days a closure is required for
lambda objects because of the name binding mechanism.

e.g. (in pseudo lisp/tcl)
proc compose {g f} {
return [lambda {x} {$g [$f $x]}]
}

In lisp the $g and $f would be bound to the g and f parameters to
compose. In Tcl however $g and $f would be bound to local variables
inside lambda so this would not work. The Feather version of compose
would do something like

proc compose {g f} {
return [curry [lambda {g f x} {$g [$f $x]}] $g $f]
}

which is what I think happens in languages like ML (if there is one like
it) although they have built in currying.

Having said all of that is there any need for closure in Tcl ?

Christopher Nelson

unread,
May 6, 1999, 3:00:00 AM5/6/99
to
Alexandre Ferrieux wrote:

>
> Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
> >
> > Right, I totally agree.
> > Maybe there is a friend of TclProcCompileProc that can be used, so one can
> > provide a command 'lambda' that returns a precompiled anonymous function?
>
> As I was wondering in a nearby post, do you know if currently the
> compiled bytecode of the source in $x is thrown away after each
> iteration of:
>
> set x "some code"
> while {1} {eval $x}
>

This relates to a question about the bytecode compiler that I've never had
answered:

In a "real" compiler, an optimizer can look at the construct above and say "Ah,
x isn't set in the while so I can't change this to:

while {1} { eval "some code" }

There are lots of other optimizations like:

set x 2
for {set i 1} {$i < 10} { incr i} {
puts [expr $x * $i]
}

which first becomes:

for {set i 1} {$i < 10} { incr i} {
puts [expr 2 * $i]
}

and than can become:

for {set i 1} {$i < 10} { incr i} {
puts [expr $i << 1]
}

And, ultimately:

puts [expr 1 << 1]
puts [expr 2 << 1]
puts [expr 3 << 1]
puts [expr 4 << 1]
puts [expr 5 << 1]
puts [expr 6 << 1]
puts [expr 7 << 1]
puts [expr 8 << 1]
puts [expr 9 << 1]

Does Tcl do any of these optimizations? *Can* a scripting language do any of
these optimizations? I can see problems with the first case where "some code"
affects x but I'm not sure they are insurmountable.

Chris
--
Rens-se-LEER is a county. RENS-se-ler is a city. R-P-I is a school!

Bruce S. O. Adams

unread,
May 6, 1999, 3:00:00 AM5/6/99
to

Christopher Nelson wrote:

I suspect that only a handful of possible optimizations are carried out at the
momment.
The trouble is knowing which type to do and when. There is bound to be a
performance
trade off somewhere. It would be interesting to have user level access to byte
code so
that we could play around with such things ourselves. It might prove to be an
interesting
level between pure TCL and C. On the other hand, if you are having performance
problems
you're really encouraged to rewrite things in C. I still like the idea though.
Regards,
Bruce A.

Donal K. Fellows

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
In article <3731A47B...@pinebush.com>,

Christopher Nelson <ch...@pinebush.com> wrote:
> In a "real" compiler, an optimizer can look at the construct above
> and say "Ah, x isn't set in the while so I can't change this to:
> while {1} { eval "some code" }
> There are lots of other optimizations like:
> set x 2
> for {set i 1} {$i < 10} { incr i} {
> puts [expr $x * $i]
> }
[...]

Slight problem: read and write traces. :^)

Donal K. Fellows

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
In article <373175...@mailserver.hursley.ibm.com>,
Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
> Alexandre Ferrieux wrote:
>> http://place.net/~af/tcl/curry.tcl

>
> Please place it in its own namespace, if you wanted to add it to the
> core then make sure that it goes in ::tcl:: namespace.

*nod*nod*

> I have not exhaustively tested it so do not know exactly what it will
> break, however it not only breaks if proc names have spaces, it also
> breaks if namespaces have spaces too.

It is also fragile (think "unquoted metacharacters") if you don't use
[lambda] to create the code. Forcing everyone to use [lambda] is
probably a good thing though.

Donal K. Fellows

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
In article <37319C...@mailserver.hursley.ibm.com>,
Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:

> Donal K. Fellows wrote:
>> Neat work, Alex! Next stop; closures.
>
> What does a closure mean to you from a Tcl point of view ?

A bit like an anonymous namespace.

> If I remember correctly from my lisp days a closure is required for
> lambda objects because of the name binding mechanism.
> e.g. (in pseudo lisp/tcl)
> proc compose {g f} {
> return [lambda {x} {$g [$f $x]}]
> }
> In lisp the $g and $f would be bound to the g and f parameters to
> compose. In Tcl however $g and $f would be bound to local variables
> inside lambda so this would not work.

Why? You're assuming that [lambda] has the same variable binding
semantics of [proc]. With true closures, the only variable binding
that [lambda] would introduce would be for its own variables. So in:

proc compose {g f} {
return [lambda {x} {set y [$f $x]; $g $y}]
}

The variable y is at the scope of the body of the compose procedure.
But any variables defined in whatever $f and $g are will not be at
that scope. Perl gives closures with the "my" operator.

> Having said all of that is there any need for closure in Tcl ?

Strictly, no, we don't need them. However, with closures you could do
away with the restrictions on the scope of variables for -textvariable
arguments to widgets, etc.

But then, I'm really a functional programmer anyway. I don't need a
reason to introduce closures!

Christopher Nelson

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
"Donal K. Fellows" wrote:
>
> In article <3731A47B...@pinebush.com>,
> Christopher Nelson <ch...@pinebush.com> wrote:
> > In a "real" compiler, an optimizer can look at the construct above
> > and say "Ah, x isn't set in the while so I can't change this to:
> > while {1} { eval "some code" }
> > There are lots of other optimizations like:
> > set x 2
> > for {set i 1} {$i < 10} { incr i} {
> > puts [expr $x * $i]
> > }
> [...]
>
> Slight problem: read and write traces. :^)

Well, yes but if I turn on optimizations (tclsh -opt or set tclOptimze 1 or
something), I would expect that and code for it.

Paul Duffin

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
Bruce S. O. Adams wrote:
>
> Christopher Nelson wrote:
>
> > Alexandre Ferrieux wrote:
> > >
> > > Wolfgang S. Kechel - Patzschke + Rasp GmbH wrote:
> > > >
> > > > Right, I totally agree.
> > > > Maybe there is a friend of TclProcCompileProc that can be used, so one can
> > > > provide a command 'lambda' that returns a precompiled anonymous function?
> > >
> > > As I was wondering in a nearby post, do you know if currently the
> > > compiled bytecode of the source in $x is thrown away after each
> > > iteration of:
> > >
> > > set x "some code"
> > > while {1} {eval $x}
> > >
> >
> > This relates to a question about the bytecode compiler that I've never had
> > answered:
> >
> > In a "real" compiler, an optimizer can look at the construct above and say "Ah,
> > x isn't set in the while so I can't change this to:
> >
> > while {1} { eval "some code" }
> >

In Tcl the compiler (which is a real compiler because it converts from one
language into another language) byte compiles $x once and then uses it
again and again through the loop and anywhere else $x is used outside the
loop. If however eval is called with more than one argument then the
byte code is lost.

No optimisations are carried out along the lines that you mentioned as
far as I know.

>
> I suspect that only a handful of possible optimizations are carried out at the
> momment.
> The trouble is knowing which type to do and when. There is bound to be a
> performance
> trade off somewhere. It would be interesting to have user level access to byte
> code so
> that we could play around with such things ourselves. It might prove to be an
> interesting
> level between pure TCL and C. On the other hand, if you are having performance
> problems
> you're really encouraged to rewrite things in C. I still like the idea though.

I am currently looking at adding some new compiled commands to Tcl in my
Feather extension (in fact I already have one). I don't really think that
the benefits of doing optimisation outweigh the costs, especially as the
byte codes are quite powerful. What is beneficial is replacing searches
of tables with indexing arrays and generally improving the performance
of the algorithms in that way.

Paul Duffin

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
Donal K. Fellows wrote:
>
> In article <373175...@mailserver.hursley.ibm.com>,

> Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
> > Alexandre Ferrieux wrote:
> >> http://place.net/~af/tcl/curry.tcl
> >
> > Please place it in its own namespace, if you wanted to add it to the
> > core then make sure that it goes in ::tcl:: namespace.
>
> *nod*nod*
>
> > I have not exhaustively tested it so do not know exactly what it will
> > break, however it not only breaks if proc names have spaces, it also
> > breaks if namespaces have spaces too.
>
> It is also fragile (think "unquoted metacharacters") if you don't use
> [lambda] to create the code. Forcing everyone to use [lambda] is
> probably a good thing though.
>

Which is one reason why I though adding a [curry] command would be
useful.
e.g.
set a "a b c d"
set b "1 2 3 4"
set c "Z Y X W"

set curried "foo [list $a] [list $b] [list $c]"

or

set curried [curry foo $a $b $c]

Paul Duffin

unread,
May 7, 1999, 3:00:00 AM5/7/99
to
Donal K. Fellows wrote:
>
> > Having said all of that is there any need for closure in Tcl ?
>
> Strictly, no, we don't need them. However, with closures you could do
> away with the restrictions on the scope of variables for -textvariable
> arguments to widgets, etc.
>
> But then, I'm really a functional programmer anyway. I don't need a
> reason to introduce closures!
>

At least you are honest !!

Donal K. Fellows

unread,
May 10, 1999, 3:00:00 AM5/10/99
to
In article <37331A...@mailserver.hursley.ibm.com>,
Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:

> Donal K. Fellows wrote:
>> But then, I'm really a functional programmer anyway. I don't need a
>> reason to introduce closures!
>
> At least you are honest !!

Yeah. I'm honestly part of the Enclosure Movement.

Donal (will make poor history jokes for no reason at all...)

Reply all
Reply to author
Forward
0 new messages