how to write function that returns function
Paul Graham
can tell me how in Python to write a function
of one argument x that returns a function of one
argument y that returns x+y.
Here, in Scheme, is what I want to write:
(define foo (x) (lambda (y) (+ x y)))
I found on the web a page that says I could define
this as follows:
def addn(x):
return lambda y,z=y: x+z
but I don't think this is exactly the same thing,
because it returns a function that takes a second
optional argument. That is a substantial difference.
If the Scheme function is inadvertently called
(e.g. in someone else's code) with two arguments, it
would signal an error, whereas the code above would
quietly give the wrong answer.
I would appreciate it if someone could tell me the
standard way to write this so that it returns a
function of exactly one argument.
Thanks!
[To reply to me directly please use pg@bug<remove>bear.com,
removing the <remove>, because I don't check sp...@bugbear.com.]
Ian Bicking
> I found on the web a page that says I could define
> this as follows:
>
> def addn(x):
> return lambda y,z=y: x+z
>
> but I don't think this is exactly the same thing,
> because it returns a function that takes a second
> optional argument. That is a substantial difference.
> If the Scheme function is inadvertently called
> (e.g. in someone else's code) with two arguments, it
> would signal an error, whereas the code above would
> quietly give the wrong answer.
This is the typical way of doing it, and yes, it is somewhat flawed.
In Python 2.1+ you can do:
from __future__ import nested_scopes
def addn(x):
return lambda y: x+y
Ian
Bengt Richter
>I am not a Python expert, and I'm hoping someone
>can tell me how in Python to write a function
>of one argument x that returns a function of one
>argument y that returns x+y.
>
>Here, in Scheme, is what I want to write:
>
>(define foo (x) (lambda (y) (+ x y)))
>
With nested scopes (default in 2.2, import nested_scopes from __future__ in 2.1),
you can do this:
>>> def foo(x): return lambda y: x+y
...
>>> f = foo(7)
>>> f(3)
10
>>> import dis <-- handy disassembler for seeing what functions do etc.
>>> dis.dis(f)
0 SET_LINENO 1
3 LOAD_DEREF 0 (x) <-- this loads the 7 from the closure
6 LOAD_FAST 0 (y) <-- this loads the argument to the function
9 BINARY_ADD
10 RETURN_VALUE
>>> f.func_closure
(<cell at 0x008430C0: int object at 0x00795910>,)
>>> hex(id(7))
'0x795910'
Doing it as above captures x in a closure. I used a
small int 7 knowing the same instance would be used,
and we could recognize it in the closure by its id.
>I found on the web a page that says I could define
>this as follows:
>
>def addn(x):
> return lambda y,z=y: x+z
>
I don't think that's quite right. The idea is to capture x in
a way that works like the closure, and a dummy default value is
the mechanism, so I think it should be:
>>> def foo(x): return lambda y,z=x: y+z
...
>>> f = foo(7)
>>> f(3)
10
>>> dis.dis(f)
0 SET_LINENO 1
3 LOAD_FAST 0 (y) <-- this loads the first function arg
6 LOAD_FAST 1 (z) <-- this simulates the closure effect
9 BINARY_ADD \_ unless you pass a second arg
10 RETURN_VALUE
>>> f.func_closure <-- there isn't any (None)
>>> f.func_defaults
(7,)
>>> f(3,5) <-- the second arg overrides, so it's not a proper solution
8
>but I don't think this is exactly the same thing,
>because it returns a function that takes a second
>optional argument. That is a substantial difference.
>If the Scheme function is inadvertently called
>(e.g. in someone else's code) with two arguments, it
>would signal an error, whereas the code above would
>quietly give the wrong answer.
>
right
>I would appreciate it if someone could tell me the
>standard way to write this so that it returns a
>function of exactly one argument.
>
See above, but BTW, you don't have to use lambda at all:
>>> def foo(x):
... def forget_this_name(y):
... return x+y
... return forget_this_name
...
>>> f = foo(7)
>>> f(3)
10
>>> dis.dis(f)
0 SET_LINENO 2
3 SET_LINENO 3
6 LOAD_DEREF 0 (x) <-- this loads the 7 from the closure
9 LOAD_FAST 0 (y) <-- this loads the function arg
12 BINARY_ADD
13 RETURN_VALUE
14 LOAD_CONST 0 (None) <--this is dead boilerplate code you don't get with lambda
17 RETURN_VALUE
>>> f.func_closure
(<cell at 0x008430C0: int object at 0x00795910>,) <-- proper closure this time. Note hex location
>>> f.func_defaults <-- no defaults
>>> f(3,5) <-- strictly one arg, so this fails
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: forget_this_name() takes exactly 1 argument (2 given)
>>> hex(id(7))
'0x795910' <--hex location of 7
HTH
Regards,
Bengt Richter
Steve Holden
news:4f52f844.02051...@posting.google.com...
> I am not a Python expert, and I'm hoping someone
> can tell me how in Python to write a function
> of one argument x that returns a function of one
> argument y that returns x+y.
>
> Here, in Scheme, is what I want to write:
>
> (define foo (x) (lambda (y) (+ x y)))
>
> I found on the web a page that says I could define
> this as follows:
>
> def addn(x):
> return lambda y,z=y: x+z
>
> but I don't think this is exactly the same thing,
> because it returns a function that takes a second
> optional argument. That is a substantial difference.
> If the Scheme function is inadvertently called
> (e.g. in someone else's code) with two arguments, it
> would signal an error, whereas the code above would
> quietly give the wrong answer.
>
> I would appreciate it if someone could tell me the
> standard way to write this so that it returns a
> function of exactly one argument.
>
to do so. From 2.2 on you don't need top import nested_scopes from
__future__, by the way. Since functions are fisrt-class objects, it doesn't
matter what name is associated with the declarations - they can be bound to
any variable once they are returned.
Python 2.2 (#1, Dec 31 2001, 15:21:18)
[GCC 2.95.3-5 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def adder(x):
... def anyoldname(y):
... return x+y
... return anyoldname
...
>>> add3 = adder(3)
>>> add100 = adder(100)
>>> add3(12)
15
>>> add100(12)
112
>>>
Note, however, that the function name (the name bound to it in the def
statement) does get built in to its repr():
>>> add3
<function anyoldname at 0x100fffe8>
>>> add100
<function anyoldname at 0x10100278>
>>>
Hope this helps.
regards
Steve
--
-----------------------------------------------------------------------
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
-----------------------------------------------------------------------
Fernando Rodríguez
>I am not a Python expert, and I'm hoping someone
>can tell me how in Python to write a function
>of one argument x that returns a function of one
>argument y that returns x+y.
>
>Here, in Scheme, is what I want to write:
>
>(define foo (x) (lambda (y) (+ x y)))
>
>I found on the web a page that says I could define
>this as follows:
>
>def addn(x):
> return lambda y,z=y: x+z
Get a recent version of Python that includes 'nested scopes' a la Lisp, and
try:
>>> from __future__ import nested_scopes # Turn on nested scopes
>>> def f(x):
def g(y):
return y +x
return g
>>> adder = f(3)
>>> adder(4)
7
BTW, this is not exactly the same behavior you find in Lisp. For example:
x = 0 #Global variable
def f():
x = 3 # Doesnt create a dynamic binding, it creates a new local variable
return x
The 'x = 3' doesn't "setf", it "let"s
I find this behavior disturbing, and I hope special variables in Arc will
behave as in Common Lisp.
Good luck with Arc! :-)
-----------------------
Fernando Rodriguez
Paul Graham
Python scope rules have changed recently, and my info was out of
date. I am still uncertain about a couple things though: there
seem to be some restrictions on what you can do with lexical
variables and also what you can put in a lambda. Can some Python
expert tell me how you would express the Common Lisp
(defun foo (n) #'(lambda () (incf n)))
in Python?
Many thanks, --pg
Michael Hudson
You can't directly translate that. Access to variable of enclosing
scopes is read only (except in the case of global statements, but that
doesn't help here).
The usual response to questions like this is "what is it you're
actually trying to *do*?"; there may well be a natural way of
acheiving *that* despite the fact that there are no non-horrible
direct translations of the above code (usually involving defining a
class or two...).
Cheers,
M.
--
It's a measure of how much I love Python that I moved to VA, where
if things don't work out Guido will buy a plantation and put us to
work harvesting peanuts instead. -- Tim Peters, comp.lang.python
George Demmy
Here's one way:
class foo:
def __init__(self, n):
self.n = n
def next(self):
self.n += 1
return self.n
def mkfoo(n):
f = foo(n)
return f.next
bar = foo(1)
print bar(), bar(), bar()
-> 2 3 4
I don't think that the "Guido implementation" of Python allows the
capture of state in closures the same way that you can in Scheme and
CL, though you can fake it very easily, as above.
G
George Demmy
George Demmy <gde...@layton-graphics.com> writes:
> sp...@bugbear.com (Paul Graham) writes:
>
> > Thanks to everyone who replied to my earlier question. It seems
> > Python scope rules have changed recently, and my info was out of
> > date. I am still uncertain about a couple things though: there
> > seem to be some restrictions on what you can do with lexical
> > variables and also what you can put in a lambda. Can some Python
> > expert tell me how you would express the Common Lisp
> >
> > (defun foo (n) #'(lambda () (incf n)))
> >
> > in Python?
> >
> > Many thanks, --pg
>
> Here's one way:
>
> class foo:
> def __init__(self, n):
> self.n = n
>
> def next(self):
> self.n += 1
> return self.n
>
> def mkfoo(n):
> f = foo(n)
> return f.next
>
> bar = foo(1)
bar = mkfoo(1)
> print bar(), bar(), bar()
> -> 2 3 4
G
George Demmy
CL:
(defun foo (n) #'(lambda () (incf n)))
Python:
def foo(n=0):
class bar:
def __init__(self):
self.n = n
def next(self):
self.n += 1
return self.n
return bar().next
f = foo()
print f(),f(),f()
-> 2 3 4
g = foo(1)
-> 3 4 5
Arc: ? ;)
G
David Eppstein
George Demmy <gde...@layton-graphics.com> wrote:
> CL:
>
> (defun foo (n) #'(lambda () (incf n)))
>
> Python:
>
> def foo(n=0):
> class bar:
> def __init__(self):
> self.n = n
> def next(self):
> self.n += 1
> return self.n
> return bar().next
>
> f = foo()
> print f(),f(),f()
> -> 2 3 4
> g = foo(1)
> -> 3 4 5
You seem to have an off by one error on your output:
I get 1 2 3 and 2 3 4
Anyway, as long as you're going to do that, why not:
from __future__ import generators
def foo(n=0):
def bar(n=n):
while 1:
n += 1
yield n
return bar().next
[ from earlier message by same author: ]
> I don't think that the "Guido implementation" of Python allows the
> capture of state in closures the same way that you can in Scheme and
> CL, though you can fake it very easily, as above.
I think the key difference is that Python's closures only give you read
access to the variables. If you want write access you have to
encapsulate them in something else, like a list:
def foo(n=0):
n=[n]
def bar():
n[0] += 1
return n[0]
return bar
--
David Eppstein UC Irvine Dept. of Information & Computer Science
epps...@ics.uci.edu http://www.ics.uci.edu/~eppstein/
Kragen Sitaker
>
> but I don't think this is exactly the same thing,
> because it returns a function that takes a second
> optional argument. That is a substantial difference.
This is embarrassing, which I guess is why we have nested scopes in
2.1+ so we don't have to do this any more; but here's a solution for
1.5.2:
class addn:
def __init__(self, x): self.x = x
def __call__(self, y): return self.x + y
This is a class instead of a function, but that is not a substantial
difference.
Unlike the nested-scopes case, this lets you modify the state of the
object in a straightforward way; a "counter" closure in Python is
ugly:
def counter(startvalue):
state = [startvalue]
def counter_internal():
state[0] += 1
return state[0]
return counter_internal
The class equivalent is a little better:
class counter:
def __init__(self, startvalue): self.state = startvalue
def __call__(self):
self.state = self.state + 1
return self.state
(I didn't use += because this class version will work in 1.5.2 and
earlier Pythons, which don't have +=. The nested-scopes version
doesn't.)
Paul Graham
> scopes is read only (except in the case of global statements, but that
> doesn't help here).
It looks as if the closest thing would be something like this,
which a Python expert sent me:
def foo(n):
s = [n]
def bar(i):
s[0] += i
return s[0]
return bar
Although you can't modify variables from outer scopes, you can
modify *parts* of them (which to me seems like the same thing...)
The reason this problem seems kind of artificial is that I don't
need to solve this actual problem. I heard that recent Python
versions had added more support for lexical closures, and was
curious how well Python works for the type of programming in
Structure and Interpretation of Computer Programs; this is kind
of a hello_world of that genre. (It's not a homework problem,
I swear.)
Incidentally, here is Perl 5 code someone sent me for this case:
sub foo {
my ($n) = @_;
return sub {return $n += $_[0]}}
--pg
Justin Shaw
from __future__ import nested_scopes
def addn(x)
return lambda y: x + y
will do what you want.
Justin Shaw
"Paul Graham" <sp...@bugbear.com> wrote in message
news:4f52f844.02051...@posting.google.com...
Erno Kuusela
sp...@bugbear.com (Paul Graham) writes:
| Although you can't modify variables from outer scopes, you can
| modify *parts* of them (which to me seems like the same thing...)
you can dereference variables from outer scopes, but you cannot
rebind them to point to new objects.
you can change the objects the variables point to (if they are not
immutable, of course).
x += 1 would (potentially) re-bind the variable, ie set the name to
point to a new object, and you can't do that to variables from outer
scopes.
if your object had an increment() method, you could call that.
-- erno
Oren Tirosh
> a "counter" closure in Python is ugly:
>
> def counter(startvalue):
> state = [startvalue]
> def counter_internal():
> state[0] += 1
> return state[0]
> return counter_internal
How about this:
def counter(startvalue):
def gen_count():
state = startvalue
while 1:
yield state
state += 1
return gen_count().next
Oren
Kragen Sitaker
> It looks as if the closest thing would be something like this,
> which a Python expert sent me:
>
> def foo(n):
> s = [n]
> def bar(i):
> s[0] += i
> return s[0]
> return bar
I don't think it's accurate to refer to me as a "Python expert".
Maybe someone else sent you essentially the same thing?
> Incidentally, here is Perl 5 code someone sent me for this case:
>
> sub foo {
> my ($n) = @_;
> return sub {return $n += $_[0]}}
The final } should be on the next line for normal Perl style, and the
second 'return' is gratuitous. (Neither 'return' is necessary, but
only the second one is bad style.)
Michael Hudson
> > You can't directly translate that. Access to variable of enclosing
> > scopes is read only (except in the case of global statements, but that
> > doesn't help here).
>
> It looks as if the closest thing would be something like this,
> which a Python expert sent me:
Well yeah, that's what I meant by "can't directly translate"...
> def foo(n):
> s = [n]
> def bar(i):
> s[0] += i
> return s[0]
> return bar
>
> Although you can't modify variables from outer scopes, you can
> modify *parts* of them (which to me seems like the same thing...)
Really? I think there's quite a difference.
> The reason this problem seems kind of artificial is that I don't
> need to solve this actual problem. I heard that recent Python
> versions had added more support for lexical closures, and was
> curious how well Python works for the type of programming in
> Structure and Interpretation of Computer Programs;
Ah, the real question! The answer: not well.
> this is kind of a hello_world of that genre
Well, I just swiped this from the online version of SICP:
(define (make-withdraw balance)
(lambda (amount)
(if (>= balance amount)
(begin (set! balance (- balance amount))
balance)
"Insufficient funds")))
(define W1 (make-withdraw 100))
(define W2 (make-withdraw 100))
(W1 50)
50
(W2 70)
30
(W2 40)
"Insufficient funds"
(W1 40)
10
I don't suppose you need telling that in Python you do this like so:
class Balance:
def __init__(self, balance):
self.balance = balance
def withdraw(self, amount):
if amount <= self.balance:
self.balance -= amount
return self.balance
else:
return "Insufficient funds"
>>> W1 = Balance(100)
>>> W2 = Balance(100)
>>> W1.withdraw(50)
50
>>> W2.withdraw(70)
30
>>> W2.withdraw(40)
'Insufficient funds'
>>> W1.withdraw(40)
10
? (Ignoring the fact that Python has exceptions...)
> (It's not a homework problem, I swear.)
I didn't think someone called Paul Graham who signs himself pg was
still doing homework...
> Incidentally, here is Perl 5 code someone sent me for this case:
>
> sub foo {
> my ($n) = @_;
> return sub {return $n += $_[0]}}
Yuck <wink>.
Cheers,
M.
--
This makes it possible to pass complex object hierarchies to
a C coder who thinks computer science has made no worthwhile
advancements since the invention of the pointer.
-- Gordon McMillan, 30 Jul 1998
George Demmy
> In article <wusn4t46...@hades.layton-graphics.com>,
> George Demmy <gde...@layton-graphics.com> wrote:
>
> > CL:
> >
> > (defun foo (n) #'(lambda () (incf n)))
> >
> > Python:
> >
> > def foo(n=0):
> > class bar:
> > def __init__(self):
> > self.n = n
> > def next(self):
> > self.n += 1
> > return self.n
> > return bar().next
> >
> > f = foo()
> > print f(),f(),f()
> > -> 2 3 4
> > g = foo(1)
> > -> 3 4 5
>
> You seem to have an off by one error on your output:
> I get 1 2 3 and 2 3 4
Erp. You found a random bug in my python interpreter... namely the one
in my head. I should have run the code through a more reliable one
before posting, eh?
>
> Anyway, as long as you're going to do that, why not:
>
> from __future__ import generators
>
> def foo(n=0):
> def bar(n=n):
> while 1:
> n += 1
> yield n
> return bar().next
>
I wouldn't deny that your example is of greater Pythonic virtue! :)
> [ from earlier message by same author: ]
> > I don't think that the "Guido implementation" of Python allows the
> > capture of state in closures the same way that you can in Scheme and
> > CL, though you can fake it very easily, as above.
>
> I think the key difference is that Python's closures only give you read
> access to the variables. If you want write access you have to
> encapsulate them in something else, like a list:
>
> def foo(n=0):
> n=[n]
> def bar():
> n[0] += 1
> return n[0]
> return bar
That is an interesting insight. I guess that's what the class in the
other example is doing... providing encapsulation. The list is
certainly concise!
Kindest regards,
G
Bengt Richter
<flame shields up>
>>> foo = lambda y:([x for x in [[y]]], (lambda: (x.append(x.pop()+1),x[0])[1]))[1]
>>> f=foo(20)
>>> f()
21
>>> f()
22
>>> f=foo(-4)
>>> [f() for x in 'x'*7]
[-3, -2, -1, 0, 1, 2, 3]
</flame shields up>
;-)
Regards,
Bengt Richter
Bengt Richter
Since [].append returns None, we could use short cut eval rules to prog a tiny bit more clearly:
>>> foo = lambda y:[[x for x in [[y]]], lambda:x.append(x.pop()+1) or x[0]][1]
>>> f=foo(4)
>>> [f() for x in xrange(5)]
[5, 6, 7, 8, 9]
Better yet:
>>> foo = lambda y:[lambda:x.append(x.pop()+1) or x[0] for x in [[y]]][0]
>>> f=foo(4)
>>> [f() for x in '.......']
[5, 6, 7, 8, 9, 10, 11]
Regards,
Bengt Richter
Paul Graham
"translations" for the following:
Scheme: (define (foo x)
(lambda (y) (set! x (+ x y))))
Perl: sub foo {
my ($n) = @_;
sub {$n += shift}
}
Here is a summary of the answers I got. This is
the closest thing to a direct translation:
def foo(n):
s = [n]
def bar(i):
s[0] += i
return s[0]
return bar
but it is considered ugly, and the canonical way to
do this seems to be by defining a class:
class foo:
def __init__(self, n):
self.n = n
def __call__(self, i):
self.n += i
return self.n
--pg
Fernando Pérez
> but it is considered ugly, and the canonical way to
> do this seems to be by defining a class:
>
> class foo:
> def __init__(self, n):
> self.n = n
>
> def __call__(self, i):
> self.n += i
> return self.n
^^^
I think you mean:
return self
Cheers,
f.
Bengt Richter
Boo! I misread an 'i' as a '1' -- to increment by a current arg (not 1) it should be
>>> foo = lambda y:[lambda i:x.append(x.pop()+i) or x[0] for x in [[y]]][0]
>>> f=foo(0)
>>> f(0)
0
>>> [f(x) for x in xrange(10)]
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45]
>>> f=foo(100)
>>> [f(x) for x in xrange(10)]
[100, 101, 103, 106, 110, 115, 121, 128, 136, 145]
>>> [f(1) for x in xrange(10)]
[146, 147, 148, 149, 150, 151, 152, 153, 154, 155]
Regards,
Bengt Richter
Bengt Richter
>Thanks to several people who have sent me Python
>"translations" for the following:
>
>Scheme: (define (foo x)
> (lambda (y) (set! x (+ x y))))
>
Ugly and not recommended, but as close as I've yet come:
>>> def foo(x): return [lambda y:z.append(z.pop()+y) or z[0] for z in [[x]]][0]
...
It seems to work (unlike my previous misreading of incrementing by 1 instead of i
in another example).
>>> f = foo(0)
>>> f(0)
0
>>> [f(x) for x in xrange(10)]
[0, 1, 3, 6, 10, 15, 21, 28, 36, 45]
>>> [f(1) for x in xrange(10)]
[46, 47, 48, 49, 50, 51, 52, 53, 54, 55]
>>> [f(100) for x in xrange(10)]
[155, 255, 355, 455, 555, 655, 755, 855, 955, 1055]
>Perl: sub foo {
> my ($n) = @_;
> sub {$n += shift}
> }
>
>Here is a summary of the answers I got. This is
>the closest thing to a direct translation:
>
>def foo(n):
> s = [n]
> def bar(i):
> s[0] += i
> return s[0]
> return bar
>
>but it is considered ugly, and the canonical way to
>do this seems to be by defining a class:
>
>class foo:
> def __init__(self, n):
> self.n = n
>
> def __call__(self, i):
> self.n += i
> return self.n
>
Yes, it's much clearer. But I just thought of the following,
which seems to work, and seems to me also pretty clear:
>>> def foo(n):
... def bar(i):
... bar.s += i
... return bar.s
... bar.s = n
... return bar
...
>>> f=foo(4)
>>> f(3)
7
>>> f(10)
17
The function bar is accessible via its own closure and thereby the attribute s.
So the attribute namespace seems to work like an assignable-within closure ;-)
BTW, is a function's appearing in its own closure a problem for the gc?
Regards,
Bengt Richter
David Eppstein
wrote:
> Yes, it's much clearer. But I just thought of the following,
> which seems to work, and seems to me also pretty clear:
>
> >>> def foo(n):
> ... def bar(i):
> ... bar.s += i
> ... return bar.s
> ... bar.s = n
> ... return bar
> ...
> >>> f=foo(4)
> >>> f(3)
> 7
> >>> f(10)
> 17
>
> The function bar is accessible via its own closure and thereby the
> attribute s. So the attribute namespace seems to work like an
> assignable-within closure ;-)
Neat, I like this a lot better than the s=[n] trick.
Max Ischenko
Paul Graham wrote:
> Thanks to several people who have sent me Python
> "translations" for the following:
>
> def foo(n):
> s = [n]
> def bar(i):
> s[0] += i
> return s[0]
> return bar
> but it is considered ugly, and the canonical way to
> do this seems to be by defining a class:
> class foo:
> def __init__(self, n):
> self.n = n
> def __call__(self, i):
> self.n += i
> return self.n
In case anyone interested further in subject check out
http://www.paulgraham.com/icad.html
--
panic("kmem_cache_init(): Offsets are wrong - I've been messed with!");
2.2.16 /usr/src/linux/mm/slab.c
Kragen Sitaker
FWIW closures are differently flexible than class instances; you can
have several closures that share the same state, for example, while
you can inspect the state of Python class instances (without the
debugger, which lets you do the same for closures in many Schemes).