Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
lambda in list comprehension acting funny
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  Messages 26 - 50 of 52 - Collapse all  -  Translate all to Translated (View all originals) < Older  Newer >
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Steven D'Aprano  
View profile  
 More options Jul 12 2012, 7:29 am
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 12 Jul 2012 11:29:49 GMT
Local: Thurs, Jul 12 2012 7:29 am
Subject: Re: lambda in list comprehension acting funny

On Thu, 12 Jul 2012 09:44:15 +0000, Alister wrote:
> On Wed, 11 Jul 2012 08:43:11 +0200, Daniel Fetchinson wrote:

>>> funcs = [ lambda x: x**i for i in range( 5 ) ]
[...]
> Having read Steve's explanation in the other thread (which I think has
> finally flipped the light switch on lambda for me) it only requires a
> minor change

> funcs=[ lambda x,y=i:x**y for i in range(5) ]

> although I cant actually think why this construct would be needed in
> practice, how are you actually using it

I would expect that the example given is just sample code demonstrating
the problem. A slightly more realistic case might be something like
defining a bunch of callbacks for, say, a GUI application.

For example, the good ol' calculator app, where you have ten buttons for
digits 0-9. So you might give each button a callback function that
inserts its own digit into the text field:

buttons = [make_button(name=str(i)) for i in range(10)]

That works fine. So now you go to add a callback to each one:

buttons = [make_button(name=str(i), callback=lambda: FIELD.insert(i))
           for i in range(10)]

and lo and behold, the ten buttons named "0" through "9" all insert 9.
The reason is that the callback functions aren't given the value of i,
but only the name[1] "i". By the time the callbacks are actually used,
i == 9 and all the buttons share the same value.

[1] Technically closures may not store values by name, but the principle
is sound: the function, when called later, looks up the current value of
variable i, which is not necessarily the same as it was when the closure
was originally defined.

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ian Kelly  
View profile  
 More options Jul 12 2012, 12:53 pm
Newsgroups: comp.lang.python
From: Ian Kelly <ian.g.ke...@gmail.com>
Date: Thu, 12 Jul 2012 10:53:25 -0600
Local: Thurs, Jul 12 2012 12:53 pm
Subject: Re: lambda in list comprehension acting funny
On Wed, Jul 11, 2012 at 9:59 PM, Steven D'Aprano

You know, partial does handle keyword arguments correctly:

funcs = [partial(lambda x, i: x ** i, i=i) for i in range(5)]

This might be bad practice though if there are other arguments to the
right of the keywords that the eventual caller might want to specify,
as those arguments would then effectively be keyword-only.  The error
message in that case is not particularly illuminating, either:

>>> def f(a, b, c):

...   return (a, b, c)
...
>>> g = partial(f, b=42)
>>> g(3, 14)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'b'

It would be a bit clearer if it instead noted that 'c' were
keyword-only in the 'g' argspec.

Cheers,
Ian


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Rotwang  
View profile  
 More options Jul 12 2012, 1:23 pm
Newsgroups: comp.lang.python
From: Rotwang <sg...@hotmail.co.uk>
Date: Thu, 12 Jul 2012 18:23:10 +0100
Local: Thurs, Jul 12 2012 1:23 pm
Subject: Re: lambda in list comprehension acting funny
On 12/07/2012 04:59, Steven D'Aprano wrote:

I don't think anyone has suggested this yet:

funcs = [(lambda i: lambda x: x**i)(i) for i in range(5)]

It's a bit more opaque than other solutions, but it fits in one line and
avoids defining functions with an implicit second argument, if that's
desirable for some reason.

--
I have made a thing that superficially resembles music:

http://soundcloud.com/eroneity/we-berated-our-own-crapiness


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 12 2012, 10:20 pm
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 13 Jul 2012 02:20:05 GMT
Local: Thurs, Jul 12 2012 10:20 pm
Subject: Re: lambda in list comprehension acting funny

On Thu, 12 Jul 2012 18:23:10 +0100, Rotwang wrote:
> I don't think anyone has suggested this yet:

> funcs = [(lambda i: lambda x: x**i)(i) for i in range(5)]

Oooh, nice! A factory function solution. Excellent!

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 13 2012, 12:33 am
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Thu, 12 Jul 2012 21:33:40 -0700 (PDT)
Local: Fri, Jul 13 2012 12:33 am
Subject: Re: lambda in list comprehension acting funny
On Jul 11, 11:41 am, Daniel Fetchinson <fetchin...@googlemail.com>
wrote:

Your expectations are reasonable.  Heres the equivalent in Haskell
from which python has taken comprehensions.
---------------------------------
GHCi, version 7.4.1: http://www.haskell.org/ghc/  :? for help

Prelude> let funcs = [ (\ x -> x ^ i)| i <- [0..4]]
Prelude> (funcs !! 0)(2)
1
Prelude> (funcs !! 1)(2)
2
Prelude> (funcs !! 2)(2)
4
Prelude>


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 13 2012, 2:36 am
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 13 Jul 2012 06:36:45 GMT
Local: Fri, Jul 13 2012 2:36 am
Subject: Re: lambda in list comprehension acting funny

You forget to finish that sentence.

"Your expectations are reasonable, for languages that don't have
variables which can actually vary."

*wink*

For purely functional languages like Haskell, the behaviour you show
below makes sense. Since Haskell doesn't allow variables to change their
value, once a closure sees i=1 (say), then it must *always* see i=1.

But that's not the case in Python, where the Haskell behaviour would be
unfortunate. Imagine if you did this:

VERBOSE = True

def function(arg):
    if VERBOSE:
        print("calling function with arg %r" % arg)
    process(arg)

You would expect the function to honour changes to the variable VERBOSE,
would you not? Using Python's scoping rules for closures, it does. Using
Haskell's rules, it wouldn't.

> Heres the equivalent in Haskell from which python has taken
> comprehensions.
> ---------------------------------
> GHCi, version 7.4.1: http://www.haskell.org/ghc/  :? for help

> Prelude> let funcs = [ (\ x -> x ^ i)| i <- [0..4]]
> Prelude> (funcs !! 0)(2)
> 1
> Prelude> (funcs !! 1)(2)
> 2
> Prelude> (funcs !! 2)(2)
> 4
> Prelude>

--
Steven

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 13 2012, 9:44 am
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Fri, 13 Jul 2012 06:44:59 -0700 (PDT)
Local: Fri, Jul 13 2012 9:44 am
Subject: Re: lambda in list comprehension acting funny
On Jul 13, 11:36 am, Steven D'Aprano <steve

Heres a more appropriate analog

------------------------------------------
VERBOSE = True

def function(arg):
    if VERBOSE:
       print("calling function with arg %r" % arg)
    process(arg)

def caller():
    VERBOSE = False
    function(1)

---------------------------------------------
Python semantics: function sees VERBOSE False
Haskell semantics: function sees VERBOSE True


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 13 2012, 10:45 am
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Fri, 13 Jul 2012 07:45:29 -0700 (PDT)
Local: Fri, Jul 13 2012 10:45 am
Subject: Re: lambda in list comprehension acting funny
To come back to the OPs question.

Variables can be assigned. Or they can be bound.
[C++ programmers will be familiar with the difference between
initialization and assignment]

List comprehensions are defined in terms of assignment to the local
variable rather than binding.
Hence the issue.

Below are two functions that simulate mapping (lambda x: x**i) where
the i's come from some given list

def binding_version(lst):
    if not lst: return []
    i = lst[0]
    return [(lambda x: x ** i)] + binding_version(lst[1:])

def assign_version(lst):
    ret = []
    for i in lst:
        ret.append(lambda x: x**i)
    return ret
----------------------

>>> fs= binding_version([0,1,2])
>>> fs[0](2)
1
>>> fs[1](2)
2
>>> fs[2](2)

4

>>> fs= assign_version([0,1,2])
>>> fs[0](2)
4
>>> fs[1](2)
4
>>> fs[2](2)

4

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Prasad, Ramit  
View profile  
 More options Jul 13 2012, 12:12 pm
Newsgroups: comp.lang.python
From: "Prasad, Ramit" <ramit.pra...@jpmorgan.com>
Date: Fri, 13 Jul 2012 16:12:40 +0000
Local: Fri, Jul 13 2012 12:12 pm
Subject: RE: lambda in list comprehension acting funny

> VERBOSE = True

> def function(arg):
>     if VERBOSE:
>        print("calling function with arg %r" % arg)
>     process(arg)

> def caller():
>     VERBOSE = False
>     function(1)

> ---------------------------------------------
> Python semantics: function sees VERBOSE False
> Haskell semantics: function sees VERBOSE True
>>> def caller():

...     VERBOSE = False
...     function(1)
>>> def function(arg):

...     if VERBOSE:
...        print("calling function with arg %r" % arg)
...    
>>> VERBOSE = True
>>> caller()

calling function with arg 1

I might be being OCD, but caller needs `global VERBOSE` for that to
work as you explain.

Ramit

Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 13 2012, 12:46 pm
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Fri, 13 Jul 2012 09:46:00 -0700 (PDT)
Subject: Re: lambda in list comprehension acting funny
On Jul 13, 9:12 pm, "Prasad, Ramit" <ramit.pra...@jpmorgan.com> wrote:

Ok let me restate: if python were to work that way (without the
global) we could say either
a Python chooses to have dynamic scoping of variables
or
b There is a bug in python's scoping rules

I would guess that most younger folks (who've not seen lisp and apl)
would choose b

We can say the same analogously in the context of ZF expressions when
the i leaks as it does in the OPs example.

Another tack on the same question: python 3 cleaned up the variable
leakage from inside ZFs to outside.  It missed cleaning up the leakage
from one step to next.

tl;dr version: Beware of mixing up functional and imperative
programming
Double-beware when you are a language designer


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Chris Angelico  
View profile  
 More options Jul 13 2012, 1:20 pm
Newsgroups: comp.lang.python
From: Chris Angelico <ros...@gmail.com>
Date: Sat, 14 Jul 2012 03:20:04 +1000
Local: Fri, Jul 13 2012 1:20 pm
Subject: Re: lambda in list comprehension acting funny

On Sat, Jul 14, 2012 at 2:46 AM, rusi <rustompm...@gmail.com> wrote:
> Ok let me restate: if python were to work that way (without the
> global) we could say either
> a Python chooses to have dynamic scoping of variables
> or
> b There is a bug in python's scoping rules

Or c, there's a declaration at the top:

from __future__ import variable_declarations

Like braces, it's a MASSIVE improvement to the language, and it's part
of a huge conspiracy that it isn't there by default. But all it takes
is a little future statement and you're on Python 4000!

</troll>

ChrisA
(I shouldn't post at half past three in the morning. I get stupid... er.)


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Hans Mulder  
View profile  
 More options Jul 13 2012, 1:53 pm
Newsgroups: comp.lang.python
From: Hans Mulder <han...@xs4all.nl>
Date: Fri, 13 Jul 2012 19:53:16 +0200
Local: Fri, Jul 13 2012 1:53 pm
Subject: Re: lambda in list comprehension acting funny
On 13/07/12 18:12:40, Prasad, Ramit wrote:

That would be quite different from what Rusi is after.

If you add `global VERBOSE` to `caller`, then there is only one
variable named `VERBOSE` and what `function` does, depends on
the most recent assignment to that variable.

If you remove your `global VERBOSE`, then there are two
variables by that name, one global and one local to `caller`.
In that case, there is the question of which one `function`
will use.

The function `function` refers to a variable `VERBOSE` that
isn't local.  In some programming langauages, the interpreter
would then scan the call stack at run-time, looking for a scope
where that name is defined.  It would find the local one in
`caller`.  This is known as "dynamic binding".

Other interpreters use the `VERBOSE` that was in scope at
the point in the program text where `function` was defined.
In this case, that would be the global one.  This is called
"lexical binding".

Some programming languages allow you to indicate on a per-
variable basis whether you want dynamic or lexical binding.

Python is firmly in the lexical camp.  Dynamic binding is not
available in Python, and never will be.

Hope this helps,

-- HansM


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Prasad, Ramit  
View profile  
 More options Jul 13 2012, 2:06 pm
Newsgroups: comp.lang.python
From: "Prasad, Ramit" <ramit.pra...@jpmorgan.com>
Date: Fri, 13 Jul 2012 18:06:31 +0000
Local: Fri, Jul 13 2012 2:06 pm
Subject: RE: lambda in list comprehension acting funny

But that is not what Rusi writes.
"Python semantics: function sees VERBOSE False" <- function
will not see False without the use of global.

True and a good explanation, but not what I understood
Rusi to mean.

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ian Kelly  
View profile  
 More options Jul 13 2012, 2:54 pm
Newsgroups: comp.lang.python
From: Ian Kelly <ian.g.ke...@gmail.com>
Date: Fri, 13 Jul 2012 12:54:02 -0600
Local: Fri, Jul 13 2012 2:54 pm
Subject: Re: lambda in list comprehension acting funny

I don't believe that dynamic vs. lexical binding is what rusi was
attempting to describe.  If he was, then Python and Haskell would be a
bad comparison since both are lexical.  Rather, I think what he was
trying to show was capture by reference vs. capture by value in the
context of closures.  Python uses capture by reference, and so the
upvalue is the value of that reference at the time the closure is
called.  Haskell uses capture by value, and the upvalue is the value
at the time of definition.

I've also seen the distinction described as "early" vs. "late" binding
on this list, but I'm not sure how precise that is -- I believe that
terminology more accurately describes whether method and attribute
names are looked up at compile-time or at run-time, late binding being
the feature that makes duck typing possible.

Cheers,
Ian


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Hans Mulder  
View profile  
 More options Jul 13 2012, 3:26 pm
Newsgroups: comp.lang.python
From: Hans Mulder <han...@xs4all.nl>
Date: Fri, 13 Jul 2012 21:26:03 +0200
Local: Fri, Jul 13 2012 3:26 pm
Subject: Re: lambda in list comprehension acting funny
On 13/07/12 20:54:02, Ian Kelly wrote:

> I've also seen the distinction described as "early" vs. "late" binding
> on this list, but I'm not sure how precise that is -- I believe that
> terminology more accurately describes whether method and attribute
> names are looked up at compile-time or at run-time, late binding being
> the feature that makes duck typing possible.

I think that these terms describe the problem at the start of
this thread: the OP was expecting early binding.  However, Python
does late binding, or "acts funny" as it says in the subject line.

-- HansM


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 13 2012, 10:31 pm
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Fri, 13 Jul 2012 19:31:24 -0700 (PDT)
Local: Fri, Jul 13 2012 10:31 pm
Subject: Re: lambda in list comprehension acting funny
On Jul 13, 10:53 pm, Hans Mulder <han...@xs4all.nl> wrote:

> If you add `global VERBOSE` to `caller`, then there is only one
> variable named `VERBOSE` and what `function` does, depends on
> the most recent assignment to that variable.

> If you remove your `global VERBOSE`, then there are two
> variables by that name, one global and one local to `caller`.
> In that case, there is the question of which one `function`
> will use.

Good point. See below.

Thats a good intention.  The question is whether it is uniformly
realized.

Consider the following

def foo(x):
    i = 100
    if x:
        j = [i for i in range(10)]
        return i
    else:
        return i

In python 2  two different 'i's could be returned. In python3 only one
i can be returned.
One could call it dynamic binding. Evidently Guido thinks it a bug
which is why he changed it.

The leakage of i in the OPs question is the same kind of bug.

tl;dr version:
This is 2012. Dynamic binding is considered a bug even by the lisp
community where it originated.
Lets at least call it a feature and a gotcha


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 13 2012, 11:43 pm
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 14 Jul 2012 03:43:24 GMT
Local: Fri, Jul 13 2012 11:43 pm
Subject: Re: lambda in list comprehension acting funny

On Fri, 13 Jul 2012 19:31:24 -0700, rusi wrote:
> Consider the following

> def foo(x):
>     i = 100
>     if x:
>         j = [i for i in range(10)]
>         return i
>     else:
>         return i

A simpler example:

def foo():
    i = 100
    j = [i for i in range(10)]
    return i

In Python 3, foo() returns 100; in Python 2, it returns 9.

> In python 2  two different 'i's could be returned. In python3 only one i
> can be returned.
> One could call it dynamic binding.

One could also call it a tractor, but it isn't either of those things.

The difference is whether or not list comprehensions create their own
scope. In Python 2, they don't; in Python 3, they do. Personally I don't
have an opinion as to which is better, but in neither case does this have
any thing to do with lexical versus dynamic binding.

> Evidently Guido thinks it a bug which is why he changed it.

It was a case that either list comps be changed to *not* expose their
loop variable, or generator expressions be changed *to* expose their loop
variable. The Python 2 situation where list comps and gen expressions
have opposite behaviour was unfortunate.

> The leakage of i in the OPs question is the same kind of bug.

Not at all. The OP was *deliberately* creating a closure using i -- under
those circumstances, it would be a bug if i *didn't* leak.

The OP's gotcha was:

1) he expected the closure to use early binding, where i in each function
was bound to the value of i in the enclosing scope at the moment the
function was defined;

2) but Python actually uses late binding for closures, where the value of
i in each function is set to the value of i in the enclosing scope when
the function is called.

As far as I can tell, Python always uses late binding for scopes; the
only time it does early binding is for default values of function
parameters.

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
rusi  
View profile  
 More options Jul 14 2012, 12:53 am
Newsgroups: comp.lang.python
From: rusi <rustompm...@gmail.com>
Date: Fri, 13 Jul 2012 21:53:10 -0700 (PDT)
Local: Sat, Jul 14 2012 12:53 am
Subject: Re: lambda in list comprehension acting funny
On Jul 14, 8:43 am, Steven D'Aprano <steve

You did not get the point.

Converting my example to your format:

def foo_steven(n):
    i = 100
    j = [i for i in range(n)]
    return i

$ python3
Python 3.2.3 (default, Jun 26 2012, 00:38:09)
[GCC 4.7.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> def foo_steven(n):

...     i = 100
...     j = [i for i in range(n)]
...     return i
...
>>> foo_steven(0)
100
>>> foo_steven(4)
100

$ python
Python 2.7.3rc2 (default, Apr 22 2012, 22:35:38)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo_steven(n):

...     i = 100
...     j = [i for i in range(n)]
...     return i
...

>>> foo_steven(0)
100
>>> foo_steven(3)
2

Python 2:
When n>0 comprehension scope i is returned
When n=0 function scope i is returned

Python 3: The return statement is lexically outside the comprehension
and so that outside-scope's i is returned in all cases.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 14 2012, 3:46 am
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 14 Jul 2012 07:46:45 GMT
Local: Sat, Jul 14 2012 3:46 am
Subject: Re: lambda in list comprehension acting funny

I got the point. I just thought it was unnecessarily complex and that it
doesn't demonstrate what you think it does.

Yes, we know that in Python 3, list comprehensions create their own
scope, and the loop variable does not leak.

> $ python
> Python 2.7.3rc2 (default, Apr 22 2012, 22:35:38) [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> def foo_steven(n):
> ...     i = 100
> ...     j = [i for i in range(n)]
> ...     return i
> ...
>>>> foo_steven(0)
> 100
>>>> foo_steven(3)
> 2

Yes, we know that in Python 2, list comprehensions don't create their own
scope, and consequently the list variable does leak.

> Python 2:
> When n>0 comprehension scope i is returned
> When n=0 function scope i is returned

Incorrect.

In Python 2, *there is no comprehension scope*. There is only local
scope. In Python 2, regardless of the value of n, the local variable i is
ALWAYS returned. It just happens that sometimes the local variable i is
modified by the list comprehension, and sometimes it isn't. In Python 2,
this is no more mysterious than this piece of code:

def example(n):
    i = 100
    for i in range(n):
        pass
    return i

py> example(0)
100
py> example(4)
3

If the loop doesn't execute, the loop variable isn't modified. In Python
2, it doesn't matter whether you use a for-loop or a list comprehension,
the loop variable is local to the function.

> Python 3: The return statement is lexically outside the comprehension
> and so that outside-scope's i is returned in all cases.

Correct. In Python 3, list comprehensions now match generator expressions
and introduce their own scope which does not effect the local function
scope.

Some history:

http://bugs.python.org/issue510384
http://bugs.python.org/issue1110705

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
88888 Dihedral  
View profile  
 More options Jul 14 2012, 6:31 am
Newsgroups: comp.lang.python
From: 88888 Dihedral <dihedral88...@googlemail.com>
Date: Sat, 14 Jul 2012 03:31:12 -0700 (PDT)
Local: Sat, Jul 14 2012 6:31 am
Subject: Re: lambda in list comprehension acting funny
Alister於 2012年7月12日星期四UTC+8下午5時44分15秒寫道:

Uhn, there are 5 objects in the list if not factored well to be executed in
the run time.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 14 2012, 7:29 pm
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 14 Jul 2012 23:29:02 GMT
Local: Sat, Jul 14 2012 7:29 pm
Subject: Re: lambda in list comprehension acting funny

I don't remember whether it is Javascript or PHP that uses dynamic
binding, but whichever it is, it is generally considered to be a bad
idea, at least as the default or only behaviour.

Bash is another language with dynamic binding. Some very old versions of
Lisp use dynamic binding, because it was the easiest to implement. Most
modern languages use lexical (also known as static) binding, because it
is more sensible.

Here is an illustration of the difference: suppose we have two modules,
library.py and main.py:

# library.py
x = 23
def func(y):
    return x + y

# main.py
import library
x = 1000
print func(1)

If main.py prints 24 (and it does), then Python is using lexical scoping.
But if it prints 1001 (which it doesn't), then it is using dynamic
scoping. The obvious problem with dynamic binding is that the behaviour
of a function may vary depending on where you call it.

> I don't believe that dynamic vs. lexical binding is what rusi was
> attempting to describe.  If he was, then Python and Haskell would be a
> bad comparison since both are lexical.  Rather, I think what he was
> trying to show was capture by reference vs. capture by value in the
> context of closures.  Python uses capture by reference, and so the
> upvalue is the value of that reference at the time the closure is
> called.  Haskell uses capture by value, and the upvalue is the value at
> the time of definition.

I don't think "by reference" versus "by value" are good terms to use
here, since they risk conflating the issue with "call by reference"
versus "call by value" semantics. I prefer "late" versus "early", as you
suggest below.

> I've also seen the distinction described as "early" vs. "late" binding
> on this list, but I'm not sure how precise that is -- I believe that
> terminology more accurately describes whether method and attribute names
> are looked up at compile-time or at run-time,

Not necessarily *compile* time, but the distinction is between when the
function is defined (which may at compile time, or it may be at run time)
versus when the function is called. I think that gets to the heart of the
issue, not whether the capture copies a value or a reference. At some
point, the capture must make use of the value: whether it does so via a
direct C-style memory location copy, or an object pointer, or some other
mechanism, is irrelevant. What matters is whether that value is grabbed
at the time the closure is created, or when the closure is called.

> late binding being the
> feature that makes duck typing possible.

That is conflating two entirely distinct subjects. Python 1.5 had duck-
typing but no closures, so the one certainly does not depend on the other.

Duck-typing is more a philosophy than a language feature: the language
must be typed, and the programmer must prefer to program to a protocol or
a specification rather than to membership of a type.

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Chris Angelico  
View profile  
 More options Jul 14 2012, 8:49 pm
Newsgroups: comp.lang.python
From: Chris Angelico <ros...@gmail.com>
Date: Sun, 15 Jul 2012 10:49:48 +1000
Local: Sat, Jul 14 2012 8:49 pm
Subject: Re: lambda in list comprehension acting funny
On Sun, Jul 15, 2012 at 9:29 AM, Steven D'Aprano

<steve+comp.lang.pyt...@pearwood.info> wrote:
> Not necessarily *compile* time, but the distinction is between when the
> function is defined (which may at compile time, or it may be at run time)
> versus when the function is called.

I'd treat the def/lambda statement as "compile time" and the ()
operator as "run time".

ChrisA


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven D'Aprano  
View profile  
 More options Jul 15 2012, 4:32 am
Newsgroups: comp.lang.python
From: Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>
Date: 15 Jul 2012 08:32:47 GMT
Local: Sun, Jul 15 2012 4:32 am
Subject: Re: lambda in list comprehension acting funny

On Sun, 15 Jul 2012 10:49:48 +1000, Chris Angelico wrote:
> On Sun, Jul 15, 2012 at 9:29 AM, Steven D'Aprano
> <steve+comp.lang.pyt...@pearwood.info> wrote:
>> Not necessarily *compile* time, but the distinction is between when the
>> function is defined (which may at compile time, or it may be at run
>> time) versus when the function is called.

> I'd treat the def/lambda statement as "compile time" and the () operator
> as "run time".

But function definitions occur at run time, not compile time -- they are
executable statements, not instructions to the compiler to define a
function.

For example:

py> dis("def f(x): return x+1")  # Python 3.2
  1           0 LOAD_CONST               0 (<code object f at 0xb7b57de0,
                                            file "<dis>", line 1>)
              3 MAKE_FUNCTION            0
              6 STORE_NAME               0 (f)
              9 LOAD_CONST               1 (None)
             12 RETURN_VALUE

The code object is pre-compiled at compile time, but the function and
name-binding (the "def") doesn't occur until runtime.

At compile time, Python parses the source code and turns it into byte-
code. Class and function definitions are executed at run time, the same
as any other statement.

I'm not sure if this is a difference that makes a difference or not; I
think it is, but don't know enough about how closures and scoping rules
work in other languages to be sure that it does make a difference.

--
Steven


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Chris Angelico  
View profile  
 More options Jul 15 2012, 4:44 am
Newsgroups: comp.lang.python
From: Chris Angelico <ros...@gmail.com>
Date: Sun, 15 Jul 2012 18:44:09 +1000
Local: Sun, Jul 15 2012 4:44 am
Subject: Re: lambda in list comprehension acting funny
On Sun, Jul 15, 2012 at 6:32 PM, Steven D'Aprano

<steve+comp.lang.pyt...@pearwood.info> wrote:
> At compile time, Python parses the source code and turns it into byte-
> code. Class and function definitions are executed at run time, the same
> as any other statement.

Between the parse step and the 'def' execution, a code object is
created. When you call it, that code object already exists. Nothing
else really matters, unless there's a bug in the Python optimizer or
something weird like that. The nearest thing Python _has_ to a
"compile time" is the execution of def.

ChrisA


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Terry Reedy  
View profile  
 More options Jul 15 2012, 6:27 am
Newsgroups: comp.lang.python
From: Terry Reedy <tjre...@udel.edu>
Date: Sun, 15 Jul 2012 06:27:43 -0400
Local: Sun, Jul 15 2012 6:27 am
Subject: Re: lambda in list comprehension acting funny
On 7/15/2012 4:32 AM, Steven D'Aprano wrote:

> On Sun, 15 Jul 2012 10:49:48 +1000, Chris Angelico wrote:

>> On Sun, Jul 15, 2012 at 9:29 AM, Steven D'Aprano
>> <steve+comp.lang.pyt...@pearwood.info> wrote:
>>> Not necessarily *compile* time, but the distinction is between when the
>>> function is defined (which may at compile time, or it may be at run
>>> time) versus when the function is called.

>> I'd treat the def/lambda statement as "compile time" and the () operator
>> as "run time".

> But function definitions occur at run time, not compile time -- they are
> executable statements, not instructions to the compiler to define a
> function.

The () operator is 'call time'. The main points are
a) the execution of def statements and lambda expressions and the
execution of calls happen at different run times.
b) default arg objects are calculated at def/lambda time.
c) names in def bodies and lambda expressions are resolved at call time.
and
d) people more often forget c) when thinking about lambda expressions
that for def statements.

--
Terry Jan Reedy


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Messages 26 - 50 of 52 < Older  Newer >
« Back to Discussions « Newer topic     Older topic »