Can global variable be passed into Python function?

[Sam]

I need to pass a global variable into a python function. However, the global variable does not seem to be assigned after the function ends. Is it because parameters are not passed by reference? How can I get function parameters to be passed by reference in Python?

[dieter]

Sam <light...@gmail.com> writes:

> I need to pass a global variable into a python function.

Python does not really have the concept "variable".

What appears to be a variable is in fact only the binding of an
object to a name. If you assign something to a variable,
all you do is binding a different object to the name.

Thus, if you have:

i = 1
def f(x): x = 5

f(i)

Then "i" will remain "1" and not become "5".
The effect of "x = 5" is that the name "x" gets bound to "5"
(where is formerly was bound to "1").

>However, the global variable does not seem to be assigned after the function ends. Is it because parameters are not passed by reference?

Python lacks the notion of "variable". Thus, it does not
pass variables into functions but objects.
The objects, however, get passed by reference.

>How can I get function parameters to be passed by reference in Python?

You can implement your own concept of variable, e.g. like this:

class Variable(object):
def __init__(self, value): self.value = value

The little program above then becomes:

i = Variable(1)
def f(x): x.value = 5

f(i)
i.value # this is now 5

[Peter Otten]

If the variable you want to process is always the same global variable you
can declare it:

>>> x = 0
>>> def inc_x():
... global x
... x += 1
...
>>> x
0
>>> inc_x()
>>> x
1
>>> inc_x(); inc_x()
>>> x
3

If you want to change ("rebind") varying global names you have to be
explicit:

>>> def inc(x):
... return x + 1
...
>>> x = 0
>>> x = inc(x)
>>> y = 0
>>> y = inc(y)
>>> y = inc(y)
>>> x, y
(1, 2)

Then there are ugly hacks that make your code hard to follow. Don't use
these:

>>> def inc(name):
... globals()[name] += 1
...
>>> inc("x")
>>> inc("x")
>>> x
3

>>> def inc(module, name):
... setattr(module, name, getattr(module, name) + 1)
...
>>> import __main__
>>> inc(__main__, "x")
>>> inc(__main__, "x")
>>> inc(__main__, "x")
>>> x
6

[Gary Herron]

On 02/20/2014 10:37 PM, Sam wrote:
> I need to pass a global variable into a python function. However, the global variable does not seem to be assigned after the function ends. Is it because parameters are not passed by reference? How can I get function parameters to be passed by reference in Python?

Are you passing a value *into* a function, or back *out of* the
function? These are two very different things, and your question
mentions both and seems to confuse them.

You can pass any value into a function through its parameters, whether
that value comes from a local variable, global variable, or the result
of some computation. (But consider: it is the value that is being passed
in -- not the variable that contains that value.) You can't pass values
back out from a function through assignment to any of the parameters.

The global statement is a way to allow a function to assign to a global
variable, but this would not be considered "passing" a global into a
function. It's just a way to access and assign to a global variable
from withing a function.

v = 123

def f(...):
global v # Now v refers to the global v
print(v) # Accesses and prints the global v
v = 456 # Assigns to the global v.

Gary Herron

[Steven D'Aprano]

You cannot. Python does not support either pass-by-reference or pass-by-
value calling conventions. Instead, it supports the same calling
convention as Java, Ruby and other modern languages (which confusingly
those two languages both call something different), "call by object
sharing".

To understand more about Python's calling convention, and why neither
call-by-reference nor call-by-value are appropriate, please read these
articles:

https://mail.python.org/pipermail/tutor/2010-December/080505.html
http://effbot.org/zone/call-by-object.htm

and also see the Wikipedia article:

http://en.wikipedia.org/wiki/Evaluation_strategy


If you show us a sample of your code, together of a sample of how you
expect it to work, we can suggest an alternate way to solve that problem.



--
Steven

[Jussi Piitulainen]

dieter writes:

> Sam writes:
>
> > I need to pass a global variable into a python function.
>
> Python does not really have the concept "variable".
>
> What appears to be a variable is in fact only the binding of an
> object to a name. If you assign something to a variable,
> all you do is binding a different object to the name.
>
> Thus, if you have:
>
> i = 1
> def f(x): x = 5
>
> f(i)
>
> Then "i" will remain "1" and not become "5".
> The effect of "x = 5" is that the name "x" gets bound to "5"
> (where is formerly was bound to "1").

In alleged contrast, the observable behaviour of languages that "have
variables" is the same. This is not considered confusing by the people
who insist that there are no variables in Python.

Note: I do not object to Python's choice of different preferred
terminology. Python seems to be doing fine with that. I do object to
the insistence that Python needs this different terminology because it
behaves differently from the languages where variables are called
variables. Python behaves the same.

Here's Java, where I demonstrate with both a class variable i and a
local variable j; I used String so that this same demonstration will
still serve after someone points out that Java's primite numbers are
not quite objects, which is true but irrelevant:

$ cat Var.java
public class Var {
public static String i = "1";
public static String f(String x) {
x = "5";
return x;
}
public static void main(String... args) {
String j = "2";
System.out.println("i == " + i + ", j == " + j);
System.out.println("f(i) == " + f(i) + ", " +
"f(j) == " + f(j));
System.out.println("i == " + i + ", j == " + j);
}
}
$ javac Var.java
$ java -cp . Var
i == 1, j == 2
f(i) == 5, f(j) == 5
i == 1, j == 2
$

This is C, where I again demonstrate with both a global and a local
variable, but here I leave them as ints; I think C might be a language
where the word "variable" has been used to mean something like a block
of memory, which is a different usage; still, would someone be willing
to explain the behaviour of this program by "C lacks variables":

$ cat var.c
#include <stdio.h>
int i = 1;
int f(int x) { x = 5; return x; }
main() {
int j = 2;
printf("i == %d, j == %d\n", i, j);
printf("f(i) == %d, f(j) == %d\n", f(i), f(j));
printf("i == %d, j == %d\n", i, j);
}
$ gcc -o var var.c
$ ./var
i == 1, j == 2
f(i) == 5, f(j) == 5
i == 1, j == 2
$

This is Scheme, which is where I come from; its variables behave the
same as the corresponding machinery in Python with regard to the
problem at hand:

$ cat var.scm
(define i 1)
(define (f x) (set! x 5) x)
(let ((j 2))
(display "i == ") (write i) (display ", j == ") (write j) (newline)
(display "f(i) == ") (write (f i))
(display ", f(j) == ") (write (f j)) (newline)
(display "i == ") (write i) (display ", j == ") (write j) (newline))
$ gsi var.scm
i == 1, j == 2
f(i) == 5, f(j) == 5
i == 1, j == 2
$

> > However, the global variable does not seem to be assigned after
> > the function ends. Is it because parameters are not passed by
> > reference?
>
> Python lacks the notion of "variable". Thus, it does not
> pass variables into functions but objects.
> The objects, however, get passed by reference.

I think the relevant answer is simply that i and x are different
variables. In Python terminology, I think you would have to say that
they are different names. With a slight change, you would need to
explain how i and i are different names (I think), and then you would
introduce the concept of different namespaces.

Python indeed does not pass variables (and this is a relevant), but
neither do the other languages that "have variables".

[snip]

[Marko Rauhamaa]

Jussi Piitulainen <jpii...@ling.helsinki.fi>:

> In alleged contrast, the observable behaviour of languages that "have
> variables" is the same. This is not considered confusing by the people
> who insist that there are no variables in Python.

But of course there are variables in Python:

By “frozen” we mean that all local state is retained, including the
current bindings of local variables, [...] (<URL:
http://docs.python.org/3.2/reference/simple_stmts.html
#the-yield-statement>)

The public names defined by a module are determined by checking the
module’s namespace for a variable named __all__ (<URL:
http://docs.python.org/3.2/reference/simple_stmts.html
#the-import-statement>)

It would be impossible to assign to a global variable without global
(<URL: http://docs.python.org/3.2/reference/simple_stmts.html
#the-global-statement>)

etc etc.

However, your point about "observable behavior" is key, and Python users
of all people should get the principle (as it is related to duck
typing).

> Python indeed does not pass variables (and this is a relevant), but
> neither do the other languages that "have variables".

Maybe the idea comes from the fact that you can't easily pass a
variable to a function for modification.

Consider this C function:

void make_printable(const char **ref)
{
if (!*ref)
*ref = "<NULL>";
}

which allows:

make_printable(&x);
make_printable(&s->name);
make_printable(&a[i]);


Marko

[Ned Batchelder]

On 2/21/14 2:23 AM, dieter wrote:

> Sam<light...@gmail.com> writes:
>
>> >I need to pass a global variable into a python function.

> Python does not really have the concept "variable".
>
> What appears to be a variable is in fact only the binding of an
> object to a name. If you assign something to a variable,
> all you do is binding a different object to the name.
>

Man, do I hate this idea that Python has no variables. It has variables
(names associated with values, and the values can change over the course
of the program), they just don't work the same as C or Fortran
variables. In fact, they work exactly the same as Javascript or Ruby
variables.

Python's variables are names bound to values.
http://nedbatchelder.com/text/names.html has lots more details.


--
Ned Batchelder, http://nedbatchelder.com

[Marko Rauhamaa]

Ned Batchelder <n...@nedbatchelder.com>:

> Man, do I hate this idea that Python has no variables. It has
> variables (names associated with values, and the values can change
> over the course of the program),

In classic functional programming, the values of variables can't change
but they are still called variables (even though "parameters" might be
more fitting).

> they just don't work the same as C or Fortran variables. In fact, they
> work exactly the same as Javascript or Ruby variables.

Apart from a fact that a C variable has an address (references are
first-class objects, if you will), I really don't see a difference
between a C variable and a Python variable.


Marko

[Travis Griggs]

On Feb 21, 2014, at 4:13 AM, Ned Batchelder <n...@nedbatchelder.com> wrote:

> Man, do I hate this idea that Python has no variables. It has variables (names associated with values, and the values can change over the course of the program), they just don't work the same as C or Fortran variables. In fact, they work exactly the same as Javascript or Ruby variables.

Thank you!

+11

I get tired of the “Python doesn’t have variables” line.

What makes Python variables/bindings/references/aliases/namedvalues/slots/bucketsofstuff surprising to new arrivals from other language kingdoms, is that accessing is pragmatically implicit (walks the scope tree for you) and assignment may require explicitness. IOW, for some “variables”, you have to do something explicit to make the variable you want to refer to, vary. Some might say there is a lack of symmetry. Pros and cons.

Personally, I don’t care. It’s one of those lessons you just learn as you go.

[Chris Angelico]

On Sat, Feb 22, 2014 at 4:59 AM, Travis Griggs <travis...@gmail.com> wrote:
> What makes Python variables/bindings/references/aliases/namedvalues/slots/bucketsofstuff surprising to new arrivals from other language kingdoms, is that accessing is pragmatically implicit (walks the scope tree for you) and assignment may require explicitness. IOW, for some “variables”, you have to do something explicit to make the variable you want to refer to, vary. Some might say there is a lack of symmetry. Pros and cons.
>

What you're looking at there, I think, is actually quite tangential to
the question of Python having/not having variables, and it's to do
with scoping rules. Correct me if I'm wrong.

g = "global"
c = 0
def f():
print(g) # References two globals
l = "local"
print(l) # References one global and one local
global c
c = 1 # References one global

As an alternative, Python could have asked us to declare the other way:

def f():
print(g) # References two globals
local l
l = "local"
print(l) # References one global and one local
c = 1 # References one global

It would still be the same name binding / object reference model, with
just a different keyword used to change the default behaviour of name
lookups. And the difference is comparatively slight. We get an
encouragement to use locals rather than globals, but on the flip side,
we need two keywords (global/nonlocal) where a single one (local, or
as ECMAScript puts it, "var") could handle the other case. It's a lack
of symmetry, but having worked with PHP (where you have to declare
*every* global, except for the magical superglobals, and except for
functions (which are in a completely different namespace), and except
for constants), I am very much glad that Python puts everything in
together as just "names", and then allows us to reference global names
like "print" without declarations. It's a miswart.

ChrisA

[Marko Rauhamaa]

On the question of how variables can be passed to functions, C, of
course, has the & operator and Pascal has the "var" keyword.

An analogous thing can be achieved in Python 3 (but not in Python 2, I'm
afraid). The & operator corresponds to an ad hoc property class as in
the program below (not intended to be serious).

The program demonstrates how to write a universal "swap" function that
interchanges two references.


Marko

==clip=clip=clip========================================================
#!/usr/bin/env python3

import sys

def main():
some_list = [ 'a', 'b', 'c' ]
some_dict = { 'few' : 'ding',
'good' : 'dang',
'men' : 'dong' }
some_value = 'certainly'

class SomeListElementProperty:
"""&some_list[1]"""
def get(self):
return some_list[1]
def set(self, value):
some_list[1] = value

class SomeDictEntryProperty:
"""&some_dict["men"]"""
def get(self):
return some_dict['men']
def set(self, value):
some_dict['men'] = value

class SomeValueProperty:
"""&some_value"""
def get(self):
return some_value
def set(self, value):
nonlocal some_value
some_value = value

swap(SomeListElementProperty(), SomeDictEntryProperty())
swap(SomeListElementProperty(), SomeValueProperty())

sys.stdout.write("{}\n".format(some_list))
sys.stdout.write("{}\n".format(some_dict))
sys.stdout.write("{}\n".format(some_value))

def swap(px, py):
x = px.get()
y = py.get()
px.set(y)
py.set(x)

if __name__ == '__main__':
main()
==clip=clip=clip========================================================

[Ben Finney]

Ned Batchelder <n...@nedbatchelder.com> writes:

> On 2/21/14 2:23 AM, dieter wrote:
> > Sam<light...@gmail.com> writes:
> >

> >> >I need to pass a global variable into a python function.

> > Python does not really have the concept "variable".
> >
> > What appears to be a variable is in fact only the binding of an
> > object to a name. If you assign something to a variable,
> > all you do is binding a different object to the name.
> >
>

> Man, do I hate this idea that Python has no variables.

I agree with the sentiment of you've quoted from “dieter”, above. But I
also agree with your sentiment.

I think it's misleading to use the term “variable” to describe what
Python has, because that term has an unavoidable collection of
implications in the wider programming community, and many of those
implications are plain false for Python.

I think it's misleading to claim Python “has no variables” — which is
not quite the same as what “dieter” claimed. Python has a model of
binding references to values, and the Python documentation calls these
bindings “variables” in the special case where the reference is a name.
So, in that sense, Python has variables.

> Python's variables are names bound to values.

The ideal, in my view, would be for the Python documentation to never
use the term “variable” for this concept, since it leads newcomers
predictably to false inferences that mar their understanding of Python.

The damage isn't small, since it makes a rather simple and clear concept
needlessly difficult to understand, and falsehoods learned about
Python's data model need to be carefully rooted out and repaired later
down the track.

We should, instead, consistently use “reference” and/or “binding” for
this concept, since those lead to much more accurate inferences about
what is happening.

--
\ “People always ask me, ‘Where were you when Kennedy was shot?’ |
`\ Well, I don't have an alibi.” —Emo Philips |
_o__) |
Ben Finney

[Ned Batchelder]

On 2/21/14 9:47 PM, Dennis Lee Bieber wrote:
> On Fri, 21 Feb 2014 09:59:17 -0800, Travis Griggs <travis...@gmail.com>
> declaimed the following:

>
>> On Feb 21, 2014, at 4:13 AM, Ned Batchelder <n...@nedbatchelder.com> wrote:
>>
>>> Man, do I hate this idea that Python has no variables. It has variables (names associated with values, and the values can change over the course of the program), they just don't work the same as C or Fortran variables. In fact, they work exactly the same as Javascript or Ruby variables.
>>
>> Thank you!
>>
>> +11
>>
>> I get tired of the “Python doesn’t have variables” line.
>>
>

> If one is coming from the world where "variable" means "name
> identifying a fixed location in memory" then Python does not have
> "variables".
>
> BASIC, C, FORTRAN, COBOL, Assembly... A "variable" is synonym for an
> address [a box that holds things].
>
> Even Python's mutable types (list, dictionary, for example) do not
> follow the "fixed location" metaphor. Each cell in a list merely references
> some object, and assignment to that cell changes the reference, not the
> object.
>

So we agree: variables in Python work differently than they do in those
languages.

If one is coming from the world where "variable" means "name bound to a
value", then Python does have "variables".

Javascript, Ruby, Lisp, Perl, PHP, Smalltalk, Lua, Julia... a
"variable" is a name bound to a value.

[Steven D'Aprano]

On Fri, 21 Feb 2014 07:13:25 -0500, Ned Batchelder wrote:

> On 2/21/14 2:23 AM, dieter wrote:
>> Sam<light...@gmail.com> writes:
>>
>>> >I need to pass a global variable into a python function.
>> Python does not really have the concept "variable".
>>
>> What appears to be a variable is in fact only the binding of an object
>> to a name. If you assign something to a variable, all you do is binding
>> a different object to the name.
>>
>>
> Man, do I hate this idea that Python has no variables. It has variables
> (names associated with values, and the values can change over the course
> of the program), they just don't work the same as C or Fortran
> variables. In fact, they work exactly the same as Javascript or Ruby
> variables.

I sympathise with your view. It seems quite ridiculous to claim that
Python has no variables. If it has no variables, what on earth does it
mean when we say "x = 42"?

But the very ridiculousness is what gives it the attention-grabbing power
that makes it a useful meme. "Python variables don't behave like C
variables" might be true, but it's also wish-washy and forgettable.

In my own case, I never quite got Python's name binding semantics until I
was introduced to the "Python has no variables" meme. That got my
attention long enough to listen to the actual message: my assumptions
about how variables behave was based on Pascal semantics, and Python
doesn't quite follow the same rules. Consequently, if I implicitly define
"variable" to mean "Pascal variables", as I had been, then Python has no
variables, it has these things called "name bindings".

That's when I got it.

I went through a phase where I too insisted that Python had no variables.
But then my natural laziness asserted itself, and I decided that the word
"variable" is too useful to always reject it (and besides, C- and Pascal-
like languages don't have a monopoly on the word "variable"). Now, I use
the terms "variable" or "reference" or "name binding" as I feel makes the
most sense in context, depending on my best guess of the risk of
misunderstanding or confusion.


--
Steven

[Ned Batchelder]

This is an interesting perspective, thanks.

I think it might be that the OP's question, "Can a global variable be
passed into a function", really had nothing to do with the
name/value/variable distinction, and we've done it again: taken a simple
question and spun off into pedantry and trivia.

[Steven D'Aprano]

On Fri, 21 Feb 2014 22:45:14 -0500, Ned Batchelder wrote:

> I think it might be that the OP's question, "Can a global variable be
> passed into a function", really had nothing to do with the
> name/value/variable distinction, and we've done it again: taken a simple
> question and spun off into pedantry and trivia.

You might be right, but the OP's question isn't really clear. Surely he
didn't mean something as trivial as this?

x = 1 # This is a global
some_function(x) # Passing a global into a function


His description, especially the comment about pass by reference, suggests
that there is more to it than just passing a global variable as argument
to a function. I think he is trying to *write* to the variable as well,
as an output variable, something like this:

x = 1
y = 2
some_function(x)
assert x == 99
some_function(y)
assert y == 99


Sadly, it appears to have been a drive-by question: Sam tossed a question
out the window as he drove by, and may never come back for the answer...
I hope I'm wrong, but it's been nearly 24 hours since the question was
asked and not a peep from the OP.


Sam, if you're out there reading this, please respond with more detail
about what you are trying to do!


--
Steven

[Chris Angelico]

On Sat, Feb 22, 2014 at 6:20 AM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> On the question of how variables can be passed to functions, C, of
> course, has the & operator and Pascal has the "var" keyword.

That doesn't pass a variable into a function, though. It passes the
address of that variable, and C lets you stuff something into an
address. That's not the same thing.

ChrisA

[Steven D'Aprano]

On Fri, 21 Feb 2014 21:20:12 +0200, Marko Rauhamaa wrote:

> On the question of how variables can be passed to functions, C, of
> course, has the & operator and Pascal has the "var" keyword.
>
> An analogous thing can be achieved in Python 3 (but not in Python 2, I'm
> afraid). The & operator corresponds to an ad hoc property class as in
> the program below (not intended to be serious).

I'm glad you added that comment about not being serious. But even so,
your code is extremely unPythonic and doesn't even succeed in doing what
you try to accomplish:

> The program demonstrates how to write a universal "swap" function that
> interchanges two references.

For the record, there is no need for such a universal swap function in
Python, as the right way to do it is:

x, y = y, x


But your code doesn't succeed at doing what it sets out to do. If you try
to call it like this:

py> x = 23
py> y = 42
py> swap(x, y)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in swap
AttributeError: 'int' object has no attribute 'get'

not only doesn't it swap the two variables, but it raises an exception.
Far from being a universal swap, it's merely an obfuscated function to
swap a few hard-coded local variables.


--
Steven

[Steven D'Aprano]

Be careful about conflating the implementation with the interface. As I
understand it, the interface C literally is that &x gives you the address
of x. But that's not what the var keyword is for in Pascal (although that
may be how it is implemented).

In Pascal, if you have a function or procedure:

procedure plus(a:integer, var b:integer);
begin
b := a+b;
end;


and a couple of variables in some other scope, for simplicity lets make
them global:

var
foo: int;
bar: int;

begin
foo := 23;
bar := 1;
plus(bar, foo);
writeln(foo);
end.

the output will be 24. If we could peek into procedure plus, we would see
that argument a was a *copy* of global bar, while argument b wasn't
merely a copy of foo, it actually was foo. So assigning to b inside plus
is the same as assigning to foo in the global scope.

If we added a call:

plus(2, bar);

to the main program, then inside plus argument a would have the value 2,
and argument b would now be precisely the same variable as global bar.
After the procedure returns, bar would have the value 3.

Now I daresay that under the hood, Pascal is passing the address of foo
(or bar) to the procedure plus, but inside plus you don't see that
address as the value of b. You see the value of foo (or bar).

C does not do that -- you have to manually manage the pointers yourself,
while Pascal does it for you. And Python also has nothing like that.



--
Steven

[Marko Rauhamaa]

Steven D'Aprano <steve+comp....@pearwood.info>:

> But your code doesn't succeed at doing what it sets out to do. If you try
> to call it like this:
>
> py> x = 23
> py> y = 42
> py> swap(x, y)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "<stdin>", line 2, in swap
> AttributeError: 'int' object has no attribute 'get'
>
> not only doesn't it swap the two variables, but it raises an exception.
> Far from being a universal swap, it's merely an obfuscated function to
> swap a few hard-coded local variables.

You are calling the function wrong. Imagine the function in C. There,
you'd have to do this:

x = 23;
y = 42;
swap(&x, &y);

You've left out the ampersands and gotten a "segmentation fault."

You should have done this:

x = 23
y = 42

class XP:
def get(self):
return x
def set(self, value):
nonlocal x
x = value

class YP:
def get(self):
return y
def set(self, value):
nonlocal y
y = value

swap(XP(), YP())


So we can see that Python, too, can emulate the ampersand, albeit with
some effort.


Marko

[Chris Angelico]

On Sat, Feb 22, 2014 at 6:18 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> Now I daresay that under the hood, Pascal is passing the address of foo
> (or bar) to the procedure plus, but inside plus you don't see that
> address as the value of b. You see the value of foo (or bar).
>
> C does not do that -- you have to manually manage the pointers yourself,
> while Pascal does it for you. And Python also has nothing like that.

Yep. I should have clarified that I wasn't talking about Pascal; I'm
not fluent in the language (last time I did anything at all with
Pascal was probably about ten years ago, and not much then). In C, it
strictly does what I said: & takes the address of something, *
dereferences an address. There's no way to "pass a variable" - you
have to pass the address, and that has consequences if, for instance,
you *return* an address and the variable ceases to exist. (Does Pascal
have an equivalent of that?)

And Python has no such concept, anywhere. But anything that you can
achieve in C using pointers, you can probably achieve in Python using
more complex objects.

ChrisA

[wxjm...@gmail.com]

>>> # a swapping variant
>>> def swap(a, b):
... ab = [a, b]
... ab[1], ab[0] = ab[0], ab[1]
... return ab[0], ab[1]
...
>>> a = 111
>>> id(a)
505627864
>>> b = 999
>>> id(b)
58278640
>>> a, b = swap(a, b)
>>> a, id(a)
(999, 58278640)
>>> b, id(b)
(111, 505627864)

jmf

[Chris Angelico]

On Sat, Feb 22, 2014 at 7:02 PM, <wxjm...@gmail.com> wrote:
>>>> # a swapping variant
>>>> def swap(a, b):
> ... ab = [a, b]
> ... ab[1], ab[0] = ab[0], ab[1]
> ... return ab[0], ab[1]

Provably identical to:

def swap(a, b):
return b, a

The rest is just fluff.

ChrisA

[wxjm...@gmail.com]

Right. My bad, (just wake up).

jmf

[Steven D'Aprano]

You don't even need the function call.

a, b = b, a


--
Steven

[Steven D'Aprano]

On Sat, 22 Feb 2014 18:29:02 +1100, Chris Angelico wrote:

> On Sat, Feb 22, 2014 at 6:18 PM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> Now I daresay that under the hood, Pascal is passing the address of foo
>> (or bar) to the procedure plus, but inside plus you don't see that
>> address as the value of b. You see the value of foo (or bar).
>>
>> C does not do that -- you have to manually manage the pointers
>> yourself, while Pascal does it for you. And Python also has nothing
>> like that.
>
> Yep. I should have clarified that I wasn't talking about Pascal; I'm not
> fluent in the language (last time I did anything at all with Pascal was
> probably about ten years ago, and not much then). In C, it strictly does
> what I said: & takes the address of something, * dereferences an
> address. There's no way to "pass a variable" - you have to pass the
> address, and that has consequences if, for instance, you *return* an
> address and the variable ceases to exist. (Does Pascal have an
> equivalent of that?)

Yes, Pascal has pointers as a first-class data type. Syntax is similar to
C, ^x is a pointer to x, p^ dereferences the pointer p.

> And Python has no such concept, anywhere. But anything that you can
> achieve in C using pointers, you can probably achieve in Python using
> more complex objects.

Not even that complex. Although Python doesn't do pointers, the model of
the language is such that you don't need to. Where in C or Pascal you
would pass a pointer to a record, in Python you just pass the record,
safe in the knowledge that the entire record won't be copied.

There are a few idioms which don't work as neatly in Python as in Pascal,
such as output parameter, but you don't need them. Just return a tuple.
If you insist on an output parameter, do it like this:


def func(inarg, outarg):
if inarg % 2:
outarg[0] == "even"
else:
outarg[0] == "odd"
return inarg + 1

out = [None]
x = 42

result = func(x, out)
print(out[0])


--
Steven

[Steven D'Aprano]

On Sat, 22 Feb 2014 09:28:10 +0200, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp....@pearwood.info>:
>
>> But your code doesn't succeed at doing what it sets out to do. If you
>> try to call it like this:
>>
>> py> x = 23
>> py> y = 42
>> py> swap(x, y)
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> File "<stdin>", line 2, in swap
>> AttributeError: 'int' object has no attribute 'get'
>>
>> not only doesn't it swap the two variables, but it raises an exception.
>> Far from being a universal swap, it's merely an obfuscated function to
>> swap a few hard-coded local variables.
>
> You are calling the function wrong. Imagine the function in C. There,
> you'd have to do this:

[...]


Sorry, I misunderstood you. When you called it a universal swap function,
I thought you meant a universal swap function. I didn't realise you
intended it as a demonstration of how to emulate a C idiom using overly-
complicated Python code *wink*

If you want to emulate pointers in Python, the simplest way is to use
lists as pseudo-pointers.

# think of ptr[0] as pointer dereferencing
# think of [value] as quasi "address of" operator
def swap(p, q):
p[0], q[0] = q[0], p[0]

x = ["anything"]
y = ["something"]
z = [23]

swap(x, y)
swap(x, z)

print(x[0], y[0], z[0])
=> prints "23 anything something"


But why bother to write C in Python? Python makes a really bad C, and C
makes a really bad Python.



--
Steven

[Chris Angelico]

On Sat, Feb 22, 2014 at 7:35 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
>> Yep. I should have clarified that I wasn't talking about Pascal; I'm not
>> fluent in the language (last time I did anything at all with Pascal was
>> probably about ten years ago, and not much then). In C, it strictly does
>> what I said: & takes the address of something, * dereferences an
>> address. There's no way to "pass a variable" - you have to pass the
>> address, and that has consequences if, for instance, you *return* an
>> address and the variable ceases to exist. (Does Pascal have an
>> equivalent of that?)
>
> Yes, Pascal has pointers as a first-class data type. Syntax is similar to
> C, ^x is a pointer to x, p^ dereferences the pointer p.
>

Right, I remember those now. Yes. (See how rusty I am on it? Heh.)

ChrisA

[Chris Angelico]

On Sat, Feb 22, 2014 at 7:45 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> But why bother to write C in Python? Python makes a really bad C, and C
> makes a really bad Python.

+1.

ChrisA

[Mark Lawrence]

On 22/02/2014 02:47, Dennis Lee Bieber wrote:
> BASIC, C, FORTRAN, COBOL, Assembly... A "variable" is synonym for an
> address [a box that holds things].
>

In C.

int xyz = 1;

xyz is placed in a register. What is xyz called now as it's not in memory?

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

[Marko Rauhamaa]

Mark Lawrence <bream...@yahoo.co.uk>:

> On 22/02/2014 02:47, Dennis Lee Bieber wrote:
>> BASIC, C, FORTRAN, COBOL, Assembly... A "variable" is synonym
>> for an address [a box that holds things].
>
> In C.
>
> int xyz = 1;
>
> xyz is placed in a register. What is xyz called now as it's not in
> memory?

It's still a box, just like in Python.

The difference is that while in C, the box looks like this:

<URL: http://www.daikudojo.org/Archive/daikusan/bob.le/20090314_dovetail_box/pics/DSC_0767.JPG>

in Python, it looks like this:

<URL: http://www.dejavubuffet.fi/wp-content/uploads/2012/10/korurasia31.jpg>


Marko

[Dave Angel]

Mark Lawrence <bream...@yahoo.co.uk> Wrote in message:

> On 22/02/2014 02:47, Dennis Lee Bieber wrote:
>> BASIC, C, FORTRAN, COBOL, Assembly... A "variable" is synonym for an
>> address [a box that holds things].
>>
>
> In C.
>
> int xyz = 1;
>
> xyz is placed in a register. What is xyz called now as it's not in memory?

Don't know why you'd assume it's a register. It could just as
well be nowhere. If a later reference in the same function adds
it to something else, there might not need to be any hardware
anywhere representing the value 1.

Once you turn on a C optimizer, the real existence of local values
is not assured.


--
DaveA

[Steven D'Aprano]

On Sat, 22 Feb 2014 13:03:33 -0500, Dennis Lee Bieber wrote:

> As I recall, to handle garbage collection, Apple used to use two stage
> look ups... The user variable (handle) was a reference into a table of
> handles, and each entry in that table was a reference to the real object
> out in memory. Garbage collection would move the objects around to
> compact used memory, updating the address in the table -- the user
> program never sees the object moving as its handle address never
> changed.

Yes, but that was not transparent to the user. You actually had to
explicitly use the Mac Toolbox memory routines to allocate memory, create
and destroy handles, etc.

If you just used your programming language's normal pointers, they
couldn't and wouldn't move.



--
Steven

[Steven D'Aprano]

On Sat, 22 Feb 2014 14:15:22 +0000, Mark Lawrence wrote:

> On 22/02/2014 02:47, Dennis Lee Bieber wrote:
>> BASIC, C, FORTRAN, COBOL, Assembly... A "variable" is synonym for
>> an address [a box that holds things].
>>
>>
> In C.
>
> int xyz = 1;
>
> xyz is placed in a register. What is xyz called now as it's not in
> memory?

Of course it is in memory, just not main memory, and it is still accessed
via an address. It's just that the address is something equivalent to
"Register 5" instead of "address 12345678 in RAM".

You're focusing on the wrong thing here. The distinction is not "in main
memory" versus "in a register" (or somewhere else). The distinction is
not *where* the value lives, but the semantics of what it means to
associate a name with a value.

In C or Pascal-style languages, what we might call the "fixed address"
style of variables, a variable assignment like xyz = 1 does something
like this:

- associate the name 'xyz' with some fixed location
- stuff the value 1 into that location


In Python-style languages, what we might call the "name binding" style of
variables, that same xyz = 1 means:

- find or create the object 1
- associate the name 'xyz' with that object


In implementations like Jython and IronPython, the object is even free to
move in memory while in use. But that's not the only difference. The big
difference is that in "fixed location" languages, it makes sense to talk
about the address of a *variable*. In C, you might ask for &xyz and get
123456 regardless of whether xyz is assigned the value 1, or 23, or 999.
But in Python, you can't ask for the address of a variable, only of the
address of an *object* (and even that is useless to you, as you can't do
anything with that address).




--
Steven

[Chris Angelico]

On Sun, Feb 23, 2014 at 12:39 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> In C or Pascal-style languages, what we might call the "fixed address"
> style of variables, a variable assignment like xyz = 1 does something
> like this:
>
> - associate the name 'xyz' with some fixed location
> - stuff the value 1 into that location

Kinda. In its purest sense, C is like that. When you declare "int
xyz;", the compiler allocates one machine word of space either in the
data segment (if that's at top level, or is declared static) or on the
stack (if it's local), and records that the name xyz points there. But
an optimizing C compiler is allowed to do what it likes, as long as it
maintains that name binding... and as long as any recorded address of
it remains valid. It's actually very similar to what was discussed in
another thread recently about PyPy and the id() function - the
compiler's free to have xyz exist in different places, or not exist at
all, as long as the program can't tell the difference. I don't know
whether PyPy allocates an id for everything or only when you call
id(), but if the latter, then it's exactly the same as a C compiler
with the address-of operator - if you never take the address, it
doesn't have to have one (and even if you do, it's free to fiddle with
things, unless you declare the variable volatile).

So, these days, C is becoming more like Python.

ChrisA

[Steven D'Aprano]

On Sun, 23 Feb 2014 12:50:26 +1100, Chris Angelico wrote:

> On Sun, Feb 23, 2014 at 12:39 PM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> In C or Pascal-style languages, what we might call the "fixed address"
>> style of variables, a variable assignment like xyz = 1 does something
>> like this:
>>
>> - associate the name 'xyz' with some fixed location
>> - stuff the value 1 into that location
>
> Kinda. In its purest sense, C is like that. When you declare "int xyz;",
> the compiler allocates one machine word of space either in the data
> segment (if that's at top level, or is declared static) or on the stack
> (if it's local), and records that the name xyz points there. But an
> optimizing C compiler is allowed to do what it likes, as long as it
> maintains that name binding... and as long as any recorded address of it
> remains valid.

I think that's a red herring. The semantics of the language are that it
behaves as if the variable had a fixed location. What goes on behind the
scenes is interesting but fundamentally irrelevant if you want to
understand the language itself: the compiler's implementation may give
that variable a fixed location, or not, or multiple locations, or non-
fixed, or whatever it finds useful at the time, so long as the C code you
write can assume that it is fixed. (That's like Python, which has no
pointers. The fact that the implementation of some Python interpreters
uses pointers all over the place is strictly irrelevant.)

In practice, because C compilers usually work so close to the metal, and
programmers expect there to be a very direct correspondence between the C
code you write and the machine code you get, the compiler is unlikely to
simulate fixed addresses unless there is real benefit to be gained, it is
more likely to actually use fixed addresses. But in principle, one might
write a C interpreter in Jython, and simulate fixed addresses over the
top of a language without any addresses at all (Python), which in turn is
written in a language where the garbage collector can move things around
(Java).

The important thing here is not so much that we are disagreeing, but that
we are talking about two different levels of explanation. ("Gödel, Escher
And Bach" has an very interesting section about how explanations at
different levels can be radically different and even contradict each
other.) At the C source code level, the language mandates that variables
have fixed addresses, to the extent that C source code knows about
addresses at all. At the generated machine code level, the compiler is
free to play tricks if necessary.

Another example: if you have a variable unused=23, and the compiler
removes it because it's dead code, we wouldn't argue that therefore C
variables have no existence at all. Would we? :-)

> So, these days, C is becoming more like Python.

*raises eyebrow*

I think the stress of the exception-expression PEP is getting to you.
They truly aren't.

*wink*



--
Steven

[Chris Angelico]

On Sun, Feb 23, 2014 at 5:20 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> The important thing here is not so much that we are disagreeing, but that
> we are talking about two different levels of explanation. ("Gödel, Escher
> And Bach" has an very interesting section about how explanations at
> different levels can be radically different and even contradict each
> other.) At the C source code level, the language mandates that variables
> have fixed addresses, to the extent that C source code knows about
> addresses at all. At the generated machine code level, the compiler is
> free to play tricks if necessary.

I think that's a good summary, actually. C has variables, at the
source code level, but once you compile it down, it might not. From
which it follows that C variables have addresses (unless they're
declared 'register', in which case they might still be stored in RAM
but no longer have addresses), which may or may not have any actual
relationship to memory addresses.

> Another example: if you have a variable unused=23, and the compiler
> removes it because it's dead code, we wouldn't argue that therefore C
> variables have no existence at all. Would we? :-)
>
>> So, these days, C is becoming more like Python.
>
> *raises eyebrow*
>
> I think the stress of the exception-expression PEP is getting to you.
> They truly aren't.
>
> *wink*

Haha. I wasn't joking, though; a fully-optimizing compiler is free to
do so much that, in reality, you're not writing bare-metal code -
you're writing guidelines to a sophisticated program generator. C has
become as high level a language as any other, and has acquired some
Python-like features (check out some of the recent standards); and,
the part that I was talking about here, it makes the most sense to
talk about "name bindings" rather than memory addresses. Variables
might move around, but if you say "int xyz=5;", then you can be sure
that querying xyz will give you back 5. The concreteness of "declare a
variable ergo memory is allocated for it" is no longer the case.

ChrisA

[Marko Rauhamaa]

Steven D'Aprano <steve+comp....@pearwood.info>:

> The big difference is that in "fixed location" languages, it makes
> sense to talk about the address of a *variable*.

The address could be a symbol, too.

The Python statement

xyz = 3

places a number in the address "xyz".

You can read the value from the address "xyz" with

locals()["xyz"]


Marko

[Steven D'Aprano]

On Sun, 23 Feb 2014 11:52:05 +0200, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp....@pearwood.info>:
>
>> The big difference is that in "fixed location" languages, it makes
>> sense to talk about the address of a *variable*.
>
> The address could be a symbol, too.
>
> The Python statement
>
> xyz = 3
>
> places a number in the address "xyz".

It doesn't matter whether addresses are numeric or symbolic, Python does
not use that model for variables. Consider this example. There are a few
steps, so let's go through them. First we prove that so-called address
"abc" and "xyz" are distinct. If they were the same address, then they
would logically have to contain the same value, but that is not the case:

abc = 23
xyz = 42
assert abc != xyz


This proves that the two addresses are different.

Now we prove that they are the same address:

abc = xyz = []
xyz.append(42)
assert abc == [42]


If they were different, then modifying the object at xyz cannot modify
the object at abc. So we have a contradiction: "addresses" abc and xyz
are both the same address, and different addresses.

There is a way to wiggle out of the contradiction: accept that the
addresses are distinct, but claim that a single object can be in two
different locations at once. But if we make that argument, we are
stretching the metaphor of location quite considerably. Absent time-
travel, objects cannot be in two locations at once. Claiming that they
can be requires stretching the metaphor of location past breaking point.

Self-confession time: some years ago, I used to make exactly that
argument. I used to argue that the right way to visualise Python's
variable model was to consider that objects were stored in variables in
exactly the way you are suggesting. I didn't describe them as symbolic
addresses, but otherwise the model was the same. In order to make the
model work, I argued that objects could be in more than one place at
once, including *inside* itself:

L = []
L.append(L)

and explicitly argued that this didn't matter. After all, if I can watch
Doctor Who and accept the concept of his TARDIS materialising inside
itself, I can understand the idea of a list being inside itself.

That was before I really grasped the difference between the name binding
and fixed location variable models. While I'm still fond of the concept
of a box being inside itself, I've come to understand that having to
stretch the metaphor of location so far simply indicates that the
metaphor does not work with Python's semantics.

Python's actual behaviour is not a good fit for the "variables are
locations" model, not even if you think of locations in the abstract with
symbolic names rather the numeric addresses.



--
Steven

[Chris Angelico]

No, you cannot. That's the exact line of thinking that leads to
problems. You are not placing a number at the address "xyz", you are
pointing the name "xyz" to the number 3. That number still exists
elsewhere.

xyz = 3
abc = xyz

Does this copy the 3 from xyz into abc? In C, it would. Those
variables might not "exist" anywhere, but they must, by definition, be
storing the integer 3. And that integer has been copied. But in
Python, no it does not. It doesn't *copy* anything. And if you fiddle
with the integer 3 in some way, making it different, it'll be
different whether you look at xyz or abc, because they're the same 3.
You can't see that with integers because Python's int is immutable,
but this is where the confusion about mutable vs immutable objects
comes from. Immutable objects behave *sufficiently similarly* to the
C/Pascal model that it's possible to think Python works the same way,
but it doesn't.

The nearest C equivalent to what I'm talking about is pointers.
(CPython objects are basically used with pointers anyway. When you get
back an object reference from a CPython API function, you get a
pointer, optionally with the responsibility for one of its
references.) This is broadly how Python objects work:

/* Python strings are immutable, so the equivalent would be a list. */
/* I'm using a string because C doesn't have a list type. */
char *xyz = strcpy(malloc(20),"Hello, world!");
char *abc = xyz;

xyz[1] = 'a';
printf("abc has: %s\n", abc);

That'll show that abc has "Hallo, world!", even though it was through
xyz that the change was made. The "thing" that is that string is the
puddle of bytes on the heap (allocated with the malloc(20) up above).
The name just has a reference to that. Creating another reference to
the same object doesn't change anything.

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> That's the exact line of thinking that leads to problems. You are not
> placing a number at the address "xyz", you are pointing the name "xyz"
> to the number 3. That number still exists elsewhere.

And?

In C, I can say:

Number *o = malloc(sizeof *o);
o->value = 3;

Your statement is valid: the number 3 resides elsewhere than the
variable o.

As for Python, there's nothing in the Python specification that would
prevent you from having, say, 63-bit integers as representing
themselves. IOW, you could physically place such integers as themselves
as the reference and the number would not physically exist elsewhere.

Bottom line, there's no fundamental difference between C and Python
variables.


Marko

[Chris Angelico]

On Sun, Feb 23, 2014 at 10:01 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> As for Python, there's nothing in the Python specification that would
> prevent you from having, say, 63-bit integers as representing
> themselves. IOW, you could physically place such integers as themselves
> as the reference and the number would not physically exist elsewhere.

What would id(5) be? Some constant? What would id(id([])) be?

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

Any suitable scheme would do. For example, id(n) == n for 63-bit
integers; other objects are dynamically sequence-numbered starting from
a high base (here, 2 ** 64):

>>> id(5)
5
>>> id([])
18446744073709551620
>>> id(id([]))
18446744073709551624

Or id(n) == 2 * n for 63-bit integers; other objects are dynamically
sequence-numbered using only odd integers starting from 1:

>>> id(5)
10
>>> id([])
7
>>> id(id([]))
18

Or id(n) == 2 ** 64 + n for 63-bit integers; other objects get the
RAM address of the internal ḿemory block:

>>> id(5)
18446744073709551621
>>> id([])
3074657068
>>> id(id([]))
18446744076784207372

The possibilities are endless.


Marko

[Chris Angelico]

On Mon, Feb 24, 2014 at 2:24 AM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Or id(n) == 2 ** 64 + n for 63-bit integers; other objects get the
> RAM address of the internal ḿemory block:
>
> >>> id(5)
> 18446744073709551621
> >>> id([])
> 3074657068
> >>> id(id([]))
> 18446744076784207372

Assuming you define "63-bit integers" either as 0<=n<2**63 or as
-2**62<=n<2**62, this could work, but it would depend on never using
memory with addresses with bit 63 set, as id() is (if I recall
correctly) supposed to return an integer in the native range. I'm not
sure you can depend on that sort of pattern of memory usage.

In any case, you'd need some way to pretend that every integer is
really an object, so you'd need to define id(), the 'is' operator, and
everything else that can work with objects, to ensure that they
correctly handle this. It would be a reasonable performance
improvement to use native integers for the small ones (where "small"
integers might still be fairly large by human standards), but unlike
in languages like Pike (which does something like what you're saying),
Python has a concept of "object identity" which can't be broken.
(Pike's integers simply _are_, they aren't considered separate
objects. You can't test them for identity. Its strings, also, simply
are, although since Pike strings are guaranteed to be interned, their
values and identities really are the same. To Pike, it's only more
complex types that need to distinguish value from identity.) So this
optimization, which certainly does make sense on the face of it, would
potentially make a mess of things elsewhere.

I'm sure PyPy optimizes small integers somewhat, though.

ChrisA

[Terry Reedy]

On 2/23/2014 6:01 AM, Marko Rauhamaa wrote:

> As for Python, there's nothing in the Python specification that would
> prevent you from having, say, 63-bit integers as representing
> themselves. IOW, you could physically place such integers as themselves
> as the reference and the number would not physically exist elsewhere.

The Python spec requires that ints act as if they are independent
objects with about 20 attributes and methods. The language spec is based
on duck typing everything as an object with a class and attributes.

Revised code would have to either turn the reference int into a real
object on every access (which would be tremendously inefficient), or
code special case treatment of reference ints into *every* piece code
that might act on ints (which is all over the interpreter) so as to
simulate the int object behavior. id(someint) is the least of the problems.

Special-casing ints to store the value in the reference has been
proposed and rejected. I do not remember how far anyone went in trying
to code the idea, but I doubt that anyone got as far as getting the test
suite to pass.

> Bottom line, there's no fundamental difference between C and Python
> variables.

Hogwash. Int variables are not all variables. And as I explained above,
even if one stored Python ints much like C ints, one would have to add
code to hide the fact that one had done so.

--
Terry Jan Reedy

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> id() is (if I recall correctly) supposed to return an integer in the
> native range

That restriction seems beyond the scope of the language definition.
Still, it can be trivially provided for.

> In any case, you'd need some way to pretend that every integer is
> really an object, so you'd need to define id(), the 'is' operator, and
> everything else that can work with objects, to ensure that they
> correctly handle this.

Trivial, I should say.

> It would be a reasonable performance improvement to use native
> integers for the small ones

Maybe. Point is, whether it's done this way or that is irrelevant from
the point of view of functional correctness. Whether an integer is an
object or not has absolutely nothing to do with the implementation of
the interpreter.

And thus, Python variables are barely distinguishable from C variables.


Marko

[Marko Rauhamaa]

Terry Reedy <tjr...@udel.edu>:

> Special-casing ints to store the value in the reference has been
> proposed and rejected. I do not remember how far anyone went in trying
> to code the idea, but I doubt that anyone got as far as getting the
> test suite to pass.

FWIW, elisp essentially does that. Anyway, we are discussing this to
make it clear that the language definition is abstract. All compliant
implementations are equally correct. It doesn't make any difference
whatsoever for a Python programmer how this or that object type has been
implemented under the hood.

> Hogwash. Int variables are not all variables. And as I explained
> above, even if one stored Python ints much like C ints, one would have
> to add code to hide the fact that one had done so.

<URL: http://www.brainyquote.com/quotes/quotes/s/salvadorda103580.html>

The only difference between me and a madman is that I'm not mad.


Marko

[Mark Lawrence]

On 23/02/2014 21:04, Marko Rauhamaa wrote:
>
> And thus, Python variables are barely distinguishable from C variables.
>

To repeat what Terry Reedy said earlier, hogwash. Looks as if I've
another member of my dream team, who can proudly sit alongside our self
appointed resident unicode expert.

[Steven D'Aprano]

On Sun, 23 Feb 2014 23:10:36 +0200, Marko Rauhamaa wrote:

> Terry Reedy <tjr...@udel.edu>:
>
>> Special-casing ints to store the value in the reference has been
>> proposed and rejected. I do not remember how far anyone went in trying
>> to code the idea, but I doubt that anyone got as far as getting the
>> test suite to pass.
>
> FWIW, elisp essentially does that. Anyway, we are discussing this to
> make it clear that the language definition is abstract.

No. The language definition is concrete: it describes a concrete
interface. You are confusing the interface with the implementation: the
language definition is only abstract with regard to the implementation,
which is free to vary, so long as the interface remains the same.

Regardless of the implementation, Python code must behave in certain
ways, and the reference implementation CPython defines the semantics of
variable as name binding, not fixed locations. Can this by implemented
using fixed locations? Of course it can, and is: the proof is that
CPython's name binding variables are implemented in C, which uses fixed
location variables.

> All compliant
> implementations are equally correct. It doesn't make any difference
> whatsoever for a Python programmer how this or that object type has been
> implemented under the hood.

Performance can matter :-)

But you are correct, as far as it goes. Where you are going wrong is by
confusing the semantics of Python code with the underlying implementation
of the Python virtual machine.


--
Steven

[Gregory Ewing]

Steven D'Aprano wrote:
> Yes, Pascal has pointers as a first-class data type. Syntax is similar to
> C, ^x is a pointer to x, p^ dereferences the pointer p.

Actually, the prefix ^ is only used for declaring
pointer *types*. Standard Pascal has no way of getting
a pointer to an arbitrary variable; the only way to
get a pointer is new(p) (which is the equivalent of
p = malloc(sizeof(*p))).

--
Greg

[wxjm...@gmail.com]

Le lundi 24 février 2014 01:37:42 UTC+1, Steven D'Aprano a écrit :
>
>
>
> Performance can matter :-)
>
>
>


>>> timeit.timeit("'abc' * 1000 + 'z'")
0.991999136702321
>>> timeit.timeit("'abc' * 1000 + '\N{EURO SIGN}'")
2.5462559386176444
>>>

Two points to notice

- Even with utf-8, the worse performance case, such
a difference reaches [0.x - a few] percent.

- Indexing? Very well preserved for the "abc part"

jmf

[Steven D'Aprano]

Standard Pascal? Who uses standard Pascal? I'm talking about MacPascal :-)

Actually, it's possible that I've forgotten the details, it's been over
twenty years since I last dereferences a Pascal pointer in anger, so you
might be right... a quick glance at some of my text books suggest that
you probably are. Thanks for the correction!


--
Steven

[j.e....@gmail.com]

On Sunday, February 23, 2014 5:01:25 AM UTC-6, Marko Rauhamaa wrote:
> Chris Angelico <ros...@gmail.com>:
>
> > That's the exact line of thinking that leads to problems. You are not
>
> > placing a number at the address "xyz", you are pointing the name "xyz"
>
> > to the number 3. That number still exists elsewhere.
>
>
>
> And?
>
>
>
> In C, I can say:
>
>
>
> Number *o = malloc(sizeof *o);
>
> o->value = 3;
>
>
>
> Your statement is valid: the number 3 resides elsewhere than the
>
> variable o.

typedef struct {
int value;
} Number;

Number *o;
o = malloc(sizeof(*o));
o->value=3;
printf("o<%p>, o->value<%p>\n", o, &o->value);

o<0x9fe5008>, o->value<0x9fe5008>

Is the compiler borked?

[Michael Torrie]

On 02/24/2014 11:05 AM, j.e....@gmail.com wrote:
> typedef struct {
> int value;
> } Number;
>
> Number *o;
> o = malloc(sizeof(*o));
> o->value=3;
> printf("o<%p>, o->value<%p>\n", o, &o->value);
>
> o<0x9fe5008>, o->value<0x9fe5008>
>
> Is the compiler borked?

Why would you think that? The address of the start of your malloc'ed
structure is the same as the address of the first element. Surely this
is logical? And of course all this is quite off topic.

[rand...@fastmail.us]

On Mon, Feb 24, 2014, at 13:05, j.e....@gmail.com wrote:
> typedef struct {
> int value;
> } Number;
>
> Number *o;
> o = malloc(sizeof(*o));
> o->value=3;
> printf("o<%p>, o->value<%p>\n", o, &o->value);
>
> o<0x9fe5008>, o->value<0x9fe5008>
>
> Is the compiler borked?

That's cheating. Try printf("o<%p>", &o);

[rand...@fastmail.us]

On Mon, Feb 24, 2014, at 13:19, Michael Torrie wrote:
> Why would you think that? The address of the start of your malloc'ed
> structure is the same as the address of the first element. Surely this
> is logical? And of course all this is quite off topic.

That's not helpful - the problem, in context, is that he doesn't
understand that the fact that the structure exists at that address is
not the same thing as the variable "o" existing at that address.

[Chris Angelico]

On Tue, Feb 25, 2014 at 5:05 AM, <j.e....@gmail.com> wrote:
> typedef struct {
> int value;
> } Number;
>
> Number *o;
> o = malloc(sizeof(*o));
> o->value=3;
> printf("o<%p>, o->value<%p>\n", o, &o->value);
>
> o<0x9fe5008>, o->value<0x9fe5008>
>
> Is the compiler borked?

No, because a structure in C is simply a laying-out of its members. A
pointer to the structure is exactly the same as a pointer to its first
member, same as a pointer to an array is really just a pointer to its
first element (with other elements laid out sequentially, possibly
with padding for alignment). In Python, a complex object is a thing in
its own right; it has an identity and a set of members, each of which
has its own identity. In C, the structure is an invisible piece of
nothingness that surrounds an aggregation of other objects. For
instance:

typedef struct {
int val1;
int val2;
} NumberPair;

NumberPair five_pairs[5];
int *ten_ints = (int *)five_pairs;

This is considered naughty (pointer aliasing), and can make a mess of
optimization, but as far as I know, it's never going to be a problem
(I don't think it's possible for there to be any padding between two
ints; it's possible the struct will demand coarser alignment than the
ints would, which is why I've written them in that order). You can
access ten integers from the aliased pointer, and five pairs of
integers through the original structure array, and they're all exactly
the same. If you print out those two pointers, they will have the same
value in them, and &five_pairs[3]->val2 will be the same as
&ten_ints[7] (and the same as ten_ints+7).

For in C, neither the structure nor the non-structure is anything. The
only thing that counts is code expressing itself through love.
(Galatians 5:6, K&R translation. Or something like that.)

ChrisA

[Chris Angelico]

On Tue, Feb 25, 2014 at 5:20 AM, <rand...@fastmail.us> wrote:

> On Mon, Feb 24, 2014, at 13:05, j.e....@gmail.com wrote:
>> typedef struct {
>> int value;
>> } Number;
>>
>> Number *o;
>> o = malloc(sizeof(*o));
>> o->value=3;
>> printf("o<%p>, o->value<%p>\n", o, &o->value);
>>
>> o<0x9fe5008>, o->value<0x9fe5008>
>>
>> Is the compiler borked?
>

> That's cheating. Try printf("o<%p>", &o);

It's not cheating, but it's showing something different. Comparing o
and &o->value shows that the structure itself and its first member are
at the same location in memory. Comparing o and &o shows that the
structure and the pointer to the structure are at different locations
in memory.

ChrisA

[Mark Lawrence]

I can't be bothered to check. OTOH google groups is so please read and
action this https://wiki.python.org/moin/GoogleGroupsPython, thanks.

[Gregory Ewing]

Steven D'Aprano wrote:
> Standard Pascal? Who uses standard Pascal? I'm talking about MacPascal :-)

Mac Pascal used @ for getting a pointer to
a variable, if I remember rightly.

--
Greg

[MRAB]

So did Turbo Pascal. Delphi, which is what Turbo Pascal became, is the
same.

[Mark H. Harris]

On Friday, February 21, 2014 12:37:59 AM UTC-6, Sam wrote:
> I need to pass a global variable into a python function. However, the global variable does not seem to be assigned after the function ends. Is it because parameters are not passed by reference? How can I get function parameters to be passed by reference in Python?

def func(x):
global ref_name
ref_name = '3.14159'
# rest of the code
# rest of the code

When you call this function the ref_name reference will be set to '3.14159' as a string and your main code will be able to 'see' it, and other funcs will be able to 'see' it too... play with it a bit... if other funcs need to write to it they will also have to use the global ref_name line. As long as other funcs only read the reference, then the global line is not needed to 'see' the reference.

As others have noted, python does not have a 'variable' concept (references to objects instead) and so your question is a little ambiguous.

Also, using global references within functions is not a good idea... generally speaking.

Cheers

[Ned Batchelder]

On 2/27/14 8:24 AM, Mark H. Harris wrote:
>
> As others have noted, python does not have a 'variable' concept (references to objects instead) and so your question is a little ambiguous.
>

Mark, thanks for helping to answer the OP's question. We've covered (in
depth) in the rest of this thread, that Python *does* have the concept
of a variable, it just behaves differently than some other popular
programming languages (but exactly the same as some other popular
programming languages!)

--
Ned Batchelder, http://nedbatchelder.com

[Mark H. Harris]

On Thursday, February 27, 2014 11:54:44 AM UTC-6, Ned Batchelder wrote:

> Mark, thanks for helping to answer the OP's question. We've covered (in
> depth) in the rest of this thread, that Python *does* have the concept
> of a variable, it just behaves differently than some other popular
> programming languages (but exactly the same as some other popular
> programming languages!)
>

I do not disagree. I've had the same discussion over the years since using python, and often its a rope soaking contest... some folks get so enamored with the 'way' it works under the covers that they forget how people think about it as they are trying (as normal people) to use the language (instead of BASIC).

So, yeah, thinking about variables is just not going away.

I finally decided (in my own head) that I would completely give up on the 'variable' concept (intellectually) and help folks try to understand references and reference counting. Knowing that A points to an int, and that B=A, now B points to the VERY SAME int... they are references pointing to the same piece of memory. And of course we want new folks to understand the issue of:
A==B
True
A is B
False
..... and WHY that is or isn't....

:)

it just helps to get the 'variable' idea out of here... it really is something completely different in python.

Thanks for the note,
kind regards,
marcus

[Steven D'Aprano]

On Thu, 27 Feb 2014 15:29:01 -0800, Mark H. Harris wrote:

> Knowing that A points to an int, and
> that B=A, now B points to the VERY SAME int... they are references
> pointing to the same piece of memory. And of course we want new folks to
> understand the issue of: A==B
> True
> A is B
> False
> ..... and WHY that is or isn't....

If they point to the same piece of memory -- which, by the way, can be
moved around if the garbage collector supports it -- then A is B cannot
possibly return False.


--
Steven

[Mark H. Harris]

On Thursday, February 27, 2014 8:07:20 PM UTC-6, Steven D'Aprano wrote:

> If they point to the same piece of memory -- which, by the way, can be
> moved around if the garbage collector supports it -- then A is B cannot
> possibly return False.
>

hi Steve, long time, yes, my whole point exactly. And we all know what python is doing under the covers for small ints like 0 - 256 ... in which case consider the following:

a=128
b=a
b=128
a is b
True

But...... consider this

a=1024
b=a
b=1024
a is b
False

For small ints below a certain value (257) A is B will always be True.... but now for ints above that value, as I've shown,
A=257
B=A
B=257
A is B
False

But for the variable discussion (like in BASIC, or even C) A=128, B=A, A and B are two pieces of memory that both happen to be variables containing different pieces of memory. For python, the references to small ints (as above) will almost always point to the same int object for each reference; hence the need for reference counts.

Folks that think of assignment in python with a BASIC, or even C, mentality will have trouble understanding the difference between "==" and "is" and why... and they won't get reference counting.

Cheers

[Ben Finney]

"Mark H. Harris" <harri...@gmail.com> writes:

> So, yeah, thinking about variables is just not going away.

Right. I would like, ideally, for the Python documentation to avoid
mentioning that term entirely; and I would hope for that to promote a
better understanding of Python's data model.

The wider programming community, though, will no doubt continue to use
that term to refer to various (incompatible) data models, and I
certainly don't expect the Python community to pretend it doesn't exist.

I encourage getting rid of it from Python documentation, but not getting
rid of it from discussion in the community.

> I finally decided (in my own head) that I would completely give up on
> the 'variable' concept (intellectually) and help folks try to
> understand references and reference counting.

Reference counting isn't a concept one needs to present to newcomers,
IMO. It is sufficient to explain that the Python runtime is free to
discard an object when nothing refers to it any more.

There's no need to explain to a newcomer the garbage-collection
implementation details, precisely *because* it's an implementation
detail. Some Python implementations use reference counting, some don't,
and each implementation is free to do what it likes so long as the data
model guarantees are preserved. The user normally shouldn't care,
because they shouldn't have to depend on any specific garbage-collection
behaviour.

So: it's good to present the concept of “references”, and use “name
binding” instead of “variable”; but there's no need to present
“reference counting”, which is a Python-implementation-detail technical
concept that IMO doesn't need to be in the newcomer's head.

--
\ “The trouble with the rat race is that even if you win, you're |
`\ still a rat.” —Jane Wagner, via Lily Tomlin |
_o__) |
Ben Finney

[Chris Angelico]

On Fri, Feb 28, 2014 at 1:29 PM, Mark H. Harris <harri...@gmail.com> wrote:
> a=1024
> b=a
> b=1024
> a is b
> False

No no no no! They're not pointing to the same integer any more. Now,
if you change the "b=1024" from being a mostly-useless assignment (to
another int with the same value) into being a comparison, then it'll
be safe. But you're assigning "b=a" and then immediately reassigning
"b=1024".

Simple rule of thumb: Never use 'is' with strings or ints. They're
immutable, their identities should be their values. Playing with 'is'
will only confuse you, unless you're specifically going for
introspection and such.

ChrisA

[Mark H. Harris]

On Thursday, February 27, 2014 10:43:23 PM UTC-6, Chris Angelico wrote:

> Simple rule of thumb: Never use 'is' with strings or ints. They're
> immutable, their identities should be their values. Playing with 'is'
> will only confuse you, unless you're specifically going for
> introspection and such.

Right. The only time I use "is" is when I'm trying to explain to someone new to python assignment what is happening inside... what Mark Summerfield calls "python's beautiful heart," in his his recent book, "Programming in Python 3" ... a great resource, by the way.

[Chris Angelico]

'is' can and should be used with mutables, to distinguish between
identical values with different identities. It's also appropriate to
use 'is' with special singletons like None. Just be careful of ints
and strings.

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> Simple rule of thumb: Never use 'is' with strings or ints. They're
> immutable, their identities should be their values. Playing with 'is'
> will only confuse you, unless you're specifically going for
> introspection and such.

Here's a use case for "is" with strings (or ints):

class Connection:
IDLE = "IDLE"
CONNECTING = "CONNECTING"
CONNECTED = "CONNECTED"
DISCONNECTING = "DISCONNECTING"
DISCONNECTED = "DISCONNECTED"

def __init__(self):
self.state = IDLE

def connect(self, address):
...
self.state = CONNECTING
...

def disconnect(self):
...
if self.state is CONNECTED:
...

The state objects could have been defined like this:

IDLE = object()
CONNECTING = object()
CONNECTED = object()
DISCONNECTING = object()
DISCONNECTED = object()

However, strings have the advantage in troubleshooting:

sys.stderr.write("state = {}\n".format(self.state))


Marko

[Steven D'Aprano]

On Fri, 28 Feb 2014 09:43:58 +0200, Marko Rauhamaa wrote:

> Chris Angelico <ros...@gmail.com>:
>
>> Simple rule of thumb: Never use 'is' with strings or ints. They're
>> immutable, their identities should be their values. Playing with 'is'
>> will only confuse you, unless you're specifically going for
>> introspection and such.
>
> Here's a use case for "is" with strings (or ints):

I don't think this is a use-case for "is". See below.

> class Connection:
> IDLE = "IDLE"

[...]
> CONNECTED = "CONNECTED"
[...]

> def disconnect(self):
> ...
> if self.state is CONNECTED:
> ...

Why do you care that the state is *that specific* string, rather than any
old string with the value "CONNECTED"?

Unless you can explain a good reason why, say, *this* instance
"CONNECTED" should fail the test, while *that* instance with the same
value passes, it's not a good use-case for "is".


--
Steven

[Ben Finney]

Steven D'Aprano <st...@pearwood.info> writes:

> On Fri, 28 Feb 2014 09:43:58 +0200, Marko Rauhamaa wrote:
> > class Connection:
> > IDLE = "IDLE"
> [...]
> > CONNECTED = "CONNECTED"
> [...]
> > def disconnect(self):
> > ...
> > if self.state is CONNECTED:
> > ...
>
> Why do you care that the state is *that specific* string, rather than
> any old string with the value "CONNECTED"?

I can think of a reason:

* When you publish the API for the ‘Connection’ class,

* and another party writes code that sets ‘state’ to a string with the
value ‘"CONNECTED"’,

* and you implemented the check as ‘self.state == "CONNECTED"’,

* and their code works with your class and it goes into production,

* you're then not able to change the expected value without breaking
that party's code.

On the other hand, if the only value which will work is the one which
the caller gets via ‘Connection.CONNECTED’, then you can change the
implementation later without breaking existing code.


There are two reasons why I think this is *still* not a justification
for using ‘is’ with string values:

First reason: This is better done by making it clear the value is an
arbitrary object that won't be compared for equality. Just use
‘object()’ to creeate each value and be done with it. That's a hack, but
it's better than pretending you'll use the string as a string of text
and then breaking that expectation.

Second reason: This use case is a primary motivation for the Enum
pattern. The Enum pattern is implemented in the standard-library ‘enum’
module, now in Python 3.4 <URL:http://python.org/dev/peps/pep-0435/>.

So, I think Marko's use case is not a justification for comparing string
values with ‘is’.

--
\ “Pinky, are you pondering what I'm pondering?” “Wuh, I think |
`\ so, Brain, but if we didn't have ears, we'd look like weasels.” |
_o__) —_Pinky and The Brain_ |
Ben Finney

[Chris Angelico]

On Fri, Feb 28, 2014 at 6:43 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Here's a use case for "is" with strings (or ints):
>
> class Connection:
> IDLE = "IDLE"
> CONNECTING = "CONNECTING"
> CONNECTED = "CONNECTED"
> DISCONNECTING = "DISCONNECTING"
> DISCONNECTED = "DISCONNECTED"
>

> The state objects could have been defined like this:
>
> IDLE = object()
> CONNECTING = object()
> CONNECTED = object()
> DISCONNECTING = object()
> DISCONNECTED = object()
>
> However, strings have the advantage in troubleshooting:
>
> sys.stderr.write("state = {}\n".format(self.state))

As Ben said, strong use-case for enums (either migrate to 3.4, or
check PyPI). But here's an alternative that uses object identity
safely. (Remember, all it takes is a bit of string interning and two
equal strings could become identical.)

class enum:
def __init__(self, desc):
self.desc = desc
def __repr__(self):
return self.desc

IDLE = enum("IDLE")
CONNECTING = enum("CONNECTING")
CONNECTED = enum("CONNECTED")
DISCONNECTING = enum("DISCONNECTING")
DISCONNECTED = enum("DISCONNECTED")

Now object identity is the right way to do things, and you can still
do the formatting just like you say; plus there's no way to
accidentally get something that seems to work.

Of course, the real enum type is far more sophisticated than that, but
the concept is the same. It's an object.

ChrisA

[Marko Rauhamaa]

Ben Finney <ben+p...@benfinney.id.au>:

> There are two reasons why I think this is *still* not a justification
> for using ‘is’ with string values:
>
> First reason: This is better done by making it clear the value is an
> arbitrary object that won't be compared for equality. Just use
> ‘object()’ to creeate each value and be done with it. That's a hack,
> but it's better than pretending you'll use the string as a string of
> text and then breaking that expectation.
>
> Second reason: This use case is a primary motivation for the Enum
> pattern. The Enum pattern is implemented in the standard-library
> ‘enum’ module, now in Python 3.4
> <URL:http://python.org/dev/peps/pep-0435/>.
>
> So, I think Marko's use case is not a justification for comparing
> string values with ‘is’.

Yes, enums are long overdue. However, since any distinct objects will
do, there is nothing preventing you from using string objects.


Marko

PS On the topic of enums, when are we getting support for a switch
statement?

[Chris Angelico]

On Fri, Feb 28, 2014 at 9:02 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Yes, enums are long overdue. However, since any distinct objects will
> do, there is nothing preventing you from using string objects.
>

String literals will often be interned if they look like (especially,
if they *are*) identifiers, so if you want to prevent other strings
from happening to match, you can't trust 'is'.

>>> class Foo:
INIT = "INIT"
def __init__(self):
self.state = self.INIT
def initializing(self):
return self.state is self.INIT
>>> a=Foo()
>>> a.initializing()
True
>>> a.state="INIT"
>>> a.initializing()
True


So you should use some string value that doesn't look like an identifier:

>>> class Foo:
INIT = "<<INIT>>"
def __init__(self):
self.state = self.INIT
def initializing(self):
return self.state is self.INIT
>>> a=Foo()
>>> a.initializing()
True
>>> a.state="<<INIT>>"
>>> a.initializing()
False

But even then, chances are you can force the matter by interning explicitly.

>>> class Foo:
INIT = ">>INIT<<"
def __init__(self):
self.state = self.INIT
def initializing(self):
return self.state is self.INIT
>>> a=Foo()
>>> a.initializing()
True
>>> sys.intern(a.state)
'>>INIT<<'
>>> a.state=sys.intern(">>INIT<<")
>>> a.initializing()
True

Note that in no case did I at all tamper with the class definition,
either to change its idea of the INIT string or to fetch that
particular object. Two equal strings, in Python, might and might not
be identical, and you simply cannot rely on interning either way.

The third example, incidentally, depends on sys.intern reusing a.state
as the "one interned string". This will normally be what happens if
it's the first string of that value to be used. So you might be able
to first force the strings to be in the interning table, and then
force your sentinels to be different objects. But at that point, you
really should be using object(), or a proper enum module.

If you're using strings as state values, you should be using == to
compare them. Nothing else is safe.

ChrisA

[Chris Angelico]

On Fri, Feb 28, 2014 at 9:02 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:

> PS On the topic of enums, when are we getting support for a switch
> statement?

I don't see that they're particularly connected. In my C code, I've
used enums frequently as quick constants, often never switching on
them. Conversely, I often use switch in C code with either integer or
character constants. (ASCII character, of course, as that's all you
get.) Take this, for example, from my C++ MUD client:

enum {IS=0x00,ECHO=0x01,SEND=0x01,SUPPRESSGA=0x03,TERMTYPE=0x18,NAWS=0x1F,SE=0xF0,GA=0xF9,SB,WILL,WONT,DO=0xFD,DONT,IAC=0xFF};

That one happens to be used in a switch at one point, but it's also
used in other ways, like this:

static char naws[]={IAC,SB,NAWS,0,0,0,0,IAC,SE};

(The actual window size gets patched in separately, in case you're
curious... but I suspect most people here won't be aware that IAC SB
NAWS means Interpret-As-Command, Subnegotiation Begin, Negotiate About
Window Size, and that this is a TELNET command sequence.)

With bit flags, they'll never be used in switch:

enum //Bitflags in hackity
{
HACK_ATAT=1, //Perform @@ -> fill-in translation
HACK_AUTOEDIT=2, //Respond to the autoedit markers [now active by default]
};

...

if (hackity&HACK_ATAT) ...

Sometimes, they're just states, and they're assigned and/or compared
for equality:

enum {ic, court, citizen, trivia, sports, chancount} lastlinetype;

Some things check if (lastlinetype == ic), but nothing ever switches on it.

Meanwhile, here's a switch block from Gypsum that will never use enumerations:

switch (max(delay,base))
{
case 0..59: ... handling for <1 minute ...
case 60..3599: ... handling for <1 hour ...
default: ... handling for >= 1 hour ...
}

No, the features are quite independent. Python currently has dispatch
tables and if/elif chains, and a strong cultural aversion to switch.
You could change that by coming up with some *really* awesome
proposal, but you'll be fighting against the tide a bit.

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> String literals will often be interned if they look like (especially,
> if they *are*) identifiers, so if you want to prevent other strings
> from happening to match, you can't trust 'is'.
>

> [...]

>
> If you're using strings as state values, you should be using == to
> compare them. Nothing else is safe.

You didn't quite understand the use case. You would never ever do things
like:

> >>> a.state="<<INIT>>"

You'd only refer to the state names symbolically:

a.state = a.INIT


Marko

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> Python currently has dispatch tables and if/elif chains, and a strong
> cultural aversion to switch. You could change that by coming up with
> some *really* awesome proposal, but you'll be fighting against the
> tide a bit.

It's easy have a "cultural aversion" when the language doesn't provide
the facility.

Switch statements provide for excellent readability in parsers and state
machines, for example. They also allow the Python compiler to optimize
the statement internally unlike long if-else chains.


Marko

[Chris Angelico]

In theory, yes. If that's all people will ever do, then you can safely
use == to check. Why are you using is? To prevent the case where some
other random string will happen to compare equal. So I stuffed some
other random string in, and it was equal, and I proved that I could
make it identical as well.

ChrisA

[Chris Angelico]

On Fri, Feb 28, 2014 at 10:38 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Chris Angelico <ros...@gmail.com>:
>
>> Python currently has dispatch tables and if/elif chains, and a strong
>> cultural aversion to switch. You could change that by coming up with
>> some *really* awesome proposal, but you'll be fighting against the
>> tide a bit.
>
> It's easy have a "cultural aversion" when the language doesn't provide
> the facility.

I'm talking about the strong resistance that gets put up any time the
suggestion comes up on python-ideas or somesuch. The core devs and
Guido especially are against the idea.

> Switch statements provide for excellent readability in parsers and state
> machines, for example. They also allow the Python compiler to optimize
> the statement internally unlike long if-else chains.

It's usually possible to turn any concrete example of a switch
statement into an equally-readable dispatch table. All you need is for
your search terms to be hashable (which is a much less stringent
requirement than is usually put on switch blocks, like "must be
machine word signed integer"), and you can do something like this:

compare_key = {
# Same target(s).
ast.Assign: lambda node: ' '.join(dump(t) for t in node.targets),
# Same target and same operator.
ast.AugAssign: lambda node: dump(node.target) + dump(node.op) + "=",
# A return statement is always compatible with another.
ast.Return: lambda node: "(easy)",
# Calling these never compatible is wrong. Calling them
# always compatible will give lots of false positives.
ast.Expr: lambda node: "(maybe)",
# These ones are never compatible, so return some
# object that's never equal to anything.
ast.Import: lambda node: float("nan"),
ast.ImportFrom: lambda node: float("nan"),
ast.Pass: lambda node: float("nan"),
ast.Raise: lambda node: float("nan"),
ast.If: lambda node: float("nan"),
}

I then effectively do a big switch block like this:

if try_type not in compare_key:
print("Unrecognized type",try_type.__name__,file=sys.stderr)
compare_key[try_type] = lambda node: float("nan")
func = compare_key[try_type]
try_node = func(node.body[0])
for handler in node.handlers:
if try_node != func(handler.body[0]): return

The first check (the "not in" bit) is kinda like a default clause, but
it makes the output only once for any given type. For a more 'true'
default clause, I could do this:

try:
func = compare_key[try_type]
except KeyError:
func = compare_key["default"]

but I take advantage of the fact that the dispatch table is a dict and
mutate it. Also, if this were done in a switch block, there'd be some
redundancy. I call the same function on node.body[0] and on
handler.body[0] for each handler in handlers, so there's structure
that's common to all the branches there. I'm not sure how, with a
classic C-style switch block, I could implement that cleanly. Probably
I'd end up using function pointers and basically doing it exactly the
same way :)

The only major thing C's switch does that a dispatch table doesn't is
fall-through. And let's face it, if your big argument in favour of a
switch statement is "I need fall-through", you're not just going to
have Python devs against you, you're also going to fight against the
roughly 50% of C programmers who detest that feature :)

(FWIW, I'm in the other 50%. I quite like fall-through, and there are
times when it's a very clean way to express something. But even those
cases can usually be expressed one way or another with only a little
more redundancy - for instance, have one case that sets the key to be
the next one, and then have stand-alone if blocks rather than if/elif.
Considering that that use of fall-through usually requires an
explanatory comment anyway, you're not really losing much.)

So, going back to your statement:

> Switch statements provide for excellent readability in parsers and state
> machines, for example.

The best way to start trying to build support for this would be to
mock up a syntax for a switch statement, and find a currently-existing
parser or state machine to translate. Show us the "before and after"
shots. That's what I'm currently doing to justify an exception
expression syntax - examples like this:

pwd = (os.getcwd() except OSError: None)

# Lib/tkinter/filedialog.py:210:
try:
pwd = os.getcwd()
except OSError:
pwd = None


g = (grp.getgrnam(tarinfo.gname)[2] except KeyError: tarinfo.gid)
u = (pwd.getpwnam(tarinfo.uname)[2] except KeyError: tarinfo.uid)

# Lib/tarfile.py:2198:
try:
g = grp.getgrnam(tarinfo.gname)[2]
except KeyError:
g = tarinfo.gid
try:
u = pwd.getpwnam(tarinfo.uname)[2]
except KeyError:
u = tarinfo.uid


This is real Python code, straight out of the standard library. I
don't know if you could find many examples in the stdlib that beg for
a switch statement, but pick up some real-world code of your own, or
from some open source project, or something. Even if, in practice, it
wouldn't be changed for many years (because that project needs to be
compatible with previous versions of Python), it'd be worth showing
"here's what could be".

And then be prepared for a few hundred posts' worth of bikeshedding
about the details :)

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> On Fri, Feb 28, 2014 at 10:30 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
>> Chris Angelico <ros...@gmail.com>:

>> a.state = a.INIT
>
> In theory, yes. If that's all people will ever do, then you can safely
> use == to check. Why are you using is?

The main reason to use "is" is to indicate that the object is a sentinel
object whose identity is what is meaningful, not the content. Using ==
would work but would give some poor soul the idea that the state
variable could hold any imaginable string.

The implementation should be able to change the state objects to any
(distinct) objects at all (say, the new enums, or ints, or some more
elaborate class instances) without any changes in the code.


Marko

[Neil Cerutti]

Once you remove all the features of switch that wouldn't fit in
Python, e.g.; fall-through, jumps, breaks; whats left provides
negligible benefit over if-elif-else or dict-dispatch.

A pythonic switch statement doesn't provide enough features to
bother with its implemention.

Check out Go's switch statement for an example of what it might
look like in Python. Except you'd get it without labeled break or
the fallthrough statement. Would you still want to use it?

--
Neil Cerutti

[Marko Rauhamaa]

Neil Cerutti <ne...@norwich.edu>:

> Check out Go's switch statement for an example of what it might
> look like in Python. Except you'd get it without labeled break or
> the fallthrough statement.

No need for the fallthrough (except that multiple cases should be
supported).

Labeled breaks wouldn't be needed because there are no fallthroughs.

> Would you still want to use it?

Probably.

Guile (scheme) has:

(case (state self)
((CONNECTING CONNECTED)
...)
((DISCONNECTING)
...)
(else
...))

Python isn't "averse" to the switch statement because it would be not
that useful. Rather, the problem is that Python doesn't have nonliteral
constants (scheme has builtin symbols). It is difficult to come up with
truly Pythonic syntax for the switch statement.

Something like

switch self.state from Connection.State:
case CONNECTING or CONNECTED:
...
case DISONNECTING:
...
else:
...

would be possible, but here, "Connection.State" is evaluated at compile
time. Don't know if there are any precedents to that kind of thing in
Python.


Marko

[Neil Cerutti]

On 2014-02-28, Ben Finney <ben+p...@benfinney.id.au> wrote:
> "Mark H. Harris" <harri...@gmail.com> writes:

>> So, yeah, thinking about variables is just not going away.
>

> Right. I would like, ideally, for the Python documentation to
> avoid mentioning that term entirely; and I would hope for that
> to promote a better understanding of Python's data model.
>
> The wider programming community, though, will no doubt continue
> to use that term to refer to various (incompatible) data
> models, and I certainly don't expect the Python community to
> pretend it doesn't exist.

I like the characteristic of Python that assignment and argument
passing work the same way. If only C were so simple!

The tutorial makes things sound more high-falutin' than that
[Tutorial 4.6 Defining Functions]:

The actual parameters (arguments) to a function call are
introduced in the local symbol table of the called function
when it is called; thus, arguments are passed using call by
value (where the value is always an object reference, not the
value of the object). [...]

How about:

The actual parameters (arguments) to a function call are passed
via assignment to the variables in the local symbol table of
the called function.

Am I oversimplifying?

--
Neil Cerutti

[Chris Angelico]

On Sat, Mar 1, 2014 at 1:26 AM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Python isn't "averse" to the switch statement because it would be not
> that useful. Rather, the problem is that Python doesn't have nonliteral
> constants (scheme has builtin symbols). It is difficult to come up with
> truly Pythonic syntax for the switch statement.
>
> Something like
>
> switch self.state from Connection.State:
> case CONNECTING or CONNECTED:
> ...
> case DISONNECTING:
> ...
> else:
> ...
>
> would be possible, but here, "Connection.State" is evaluated at compile
> time. Don't know if there are any precedents to that kind of thing in
> Python.

Can you elaborate on this "nonliteral constants" point? How is it a
problem if DISCONNECTING isn't technically a constant? It follows the
Python convention of being in all upper-case, so the programmer
understands not to rebind it. Is the problem that someone might
(naively or maliciously) change the value of DISCONNECTING, or is the
problem that Python doesn't fundamentally know that it won't change?

ChrisA

[Mark Lawrence]

On 28/02/2014 11:38, Marko Rauhamaa wrote:
> Switch statements provide for excellent readability in parsers and state
> machines, for example. They also allow the Python compiler to optimize
> the statement internally unlike long if-else chains.
>

There are umpteen recipes for switch statements so take your pick or if
you don't like any of them write your own. Much easier than beating
your head against multiple brick walls, which is what raising this one
on python-ideas is likely to be. See
http://legacy.python.org/dev/peps/pep-0275/ and
http://legacy.python.org/dev/peps/pep-3103/

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

[Michael Torrie]

On 02/28/2014 01:46 AM, Ben Finney wrote:
> Steven D'Aprano <st...@pearwood.info> writes:
>

>> On Fri, 28 Feb 2014 09:43:58 +0200, Marko Rauhamaa wrote:
>>> class Connection:
>>> IDLE = "IDLE"
>> [...]
>>> CONNECTED = "CONNECTED"
>> [...]
>>> def disconnect(self):
>>> ...
>>> if self.state is CONNECTED:
>>> ...
>>
>> Why do you care that the state is *that specific* string, rather than
>> any old string with the value "CONNECTED"?
>

> I can think of a reason:
>
> * When you publish the API for the ‘Connection’ class,
>
> * and another party writes code that sets ‘state’ to a string with the
> value ‘"CONNECTED"’,
>
> * and you implemented the check as ‘self.state == "CONNECTED"’,
>
> * and their code works with your class and it goes into production,
>
> * you're then not able to change the expected value without breaking
> that party's code.

Sure. If he replaced the line if self.state is CONNECTED with if
self.state == self.CONNECTED then he is free to change CONNECTED at any
time. So yes, "is" is not necessary here. Equality checking works fine.

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> Can you elaborate on this "nonliteral constants" point? How is it a
> problem if DISCONNECTING isn't technically a constant? It follows the
> Python convention of being in all upper-case, so the programmer
> understands not to rebind it. Is the problem that someone might
> (naively or maliciously) change the value of DISCONNECTING, or is the
> problem that Python doesn't fundamentally know that it won't change?

This last point. It would make it impossible for Python to treat the
switch statement as anything but an alternate form of chained if-else.
A dict optimization wouldn't actually optimize anything because it would
have to be constructed every time the statement is executed.

switch self.state from Connection.State:
case CONNECTING or CONNECTED:
...
case DISONNECTING:
...
else:
...

would have to be transformed by Python into:

_X1 = self.state
_X2 = Connection.State
if _X1 is _X2.CONNECTING or _X1 is _X2.CONNECTED:
...
elif _X1 is _X2.DISCONNECTING:
...
else:
...

So optimization is gone. Then we have the syntactic burden. Python
currently doesn't (seem to) have a syntactic precedent for such implicit
dot notation. (Note that even Java had to complicate its syntax
analogously with enums.) In "CONNECTING or CONNECTED", "or" wouldn't be
an operator in an expression but a particle.

Another syntactic oddity is the two indentation levels.

BTW, here's a syntax that doesn't introduce any new keywords:

with self.state from Connection.State:
if CONNECTING or CONNECTED:
...
elif DISONNECTING:
...
else:
...


Marko

[Chris Angelico]

On Sat, Mar 1, 2014 at 2:29 AM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> BTW, here's a syntax that doesn't introduce any new keywords:
>
> with self.state from Connection.State:
> if CONNECTING or CONNECTED:
> ...
> elif DISONNECTING:
> ...
> else:
> ...

Okay, I understand your 'from' now. What it really does is introduce a
new scope, a read-only one presumably (because you really do NOT want
the Pandora's Box that ECMAScript's 'with' is) from which unqualified
names will be looked up. I would say that that's a very reasonable
idea, quite separately from a switch statement. Suppose you had
something like this:

with scope(Connection.State):
if self.state == CONNECTING:
print("I am not",DISCONNECTING)

It'd require a change to the LOAD_GLOBAL opcode to have it look in
multiple scopes. If you want to change something, be explicit about
where the change goes, but for lookups, it would be possible to have
them go to multiple places. I suspect, though, that this wouldn't fly;
I already posited such a theory, and was told that CPython's internals
made it much more convenient to not introduce infinitely nesting
scopes - the two use-cases that I'd most look at are these:

# This executes as a function
doubled = [x*2 for x in lst]

# This implicitly unbinds e in a finally clause
try:
foo()
except Exception as e:
pass

Neither is quite perfect; the closure method is mostly clean, but has
some extremely esoteric edge cases, and the unbinding means that a
previous value for 'e' is lost. But both are kept rather than
introducing this concept of true subscoping, because CPython's
implementation makes the latter hard.

Predicating your entire proposal on something that has been avoided
twice and just recently turned down, though, is a good way to get the
whole proposal rejected.

ChrisA

[Steven D'Aprano]

On Fri, 28 Feb 2014 12:02:03 +0200, Marko Rauhamaa wrote:

> PS On the topic of enums, when are we getting support for a switch
> statement?

http://legacy.python.org/dev/peps/pep-3103/

http://legacy.python.org/dev/peps/pep-0275/



--
Steven

[Mark H. Harris]

On Friday, February 28, 2014 9:50:09 AM UTC-6, Steven D'Aprano wrote:

> > PS On the topic of enums, when are we getting support for a switch
> > statement?

> http://legacy.python.org/dev/peps/pep-3103/
> http://legacy.python.org/dev/peps/pep-0275/
>

I have reviewed these peps, and I heard Guido's 2007 keynote, as well I have heard him speak on YouTube several times about the inadvisability of a pythonized switch statement (similar to C).

I think the real issue is about the syntax... because of python's unique indent strategy going back to ABC, a pythonized switch statement would play havoc with the many text parsers out there used for development (TestWrangler, and many others).

Personally I would still like to see a pythonized switch statement at some point. I prefer the syntactical form of PEP 275, but unlike the notion of dropping optimization and converting to if elif else under the proverbial covers, I would prefer to see a conversion to the dict dispatch table under the covers.

At any rate... and I don't think even Guido can really argue against this,... a switch statement is just more readable to human beings that a dict dispatch table, or a long if elif chain... and one of the main points of python (going all the way back to ABC) was to make very highly readable code.

marcus

[Marko Rauhamaa]

"Mark H. Harris" <harri...@gmail.com>:

> I think the real issue is about the syntax... because of python's
> unique indent strategy going back to ABC, a pythonized switch
> statement would play havoc with the many text parsers out there used
> for development (TestWrangler, and many others).

I also took a look at the proposals. I don't think it's the editor
issue. The variant I proposed most recently:

with self.state from Connection.State:
if CONNECTING or CONNECTED:
...
elif DISONNECTING:
...
else:
...

would be handled gracefully by all sane python editors, I believe.

The main problem is that it can't be optimized effectively without
bringing in an element of preprocessing. That preprocessing is done by
the human developer with the dict dispatch table, but nothing in regular
Python gives the compiler enough guaranteed information to build the
dispatch table. There are ways to be smart, but it would be a lot of
additional code for the compiler for a questionable performance gain.

> a switch statement is just more readable to human beings that a dict
> dispatch table, or a long if elif chain... and one of the main points
> of python (going all the way back to ABC) was to make very highly
> readable code.

A dict dispatch table is just awful. At least have the decency of
creating inner classes.

... which brings up the point for another post...


Marko

[Chris Angelico]

On Sat, Mar 1, 2014 at 5:53 AM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> A dict dispatch table is just awful.

Really? How is that? I've used them, often. Yes, there are times when
I could express something more cleanly with a C-style switch
statement, but other times the dispatch table is fundamentally
cleaner. I shared an example a few posts ago in this thread; care to
elaborate on how it's "just awful"?

ChrisA

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

Your example:

compare_key = {
# Same target(s).
ast.Assign: lambda node: ' '.join(dump(t) for t in node.targets),
# Same target and same operator.
ast.AugAssign: lambda node: dump(node.target) + dump(node.op) + "=",
# A return statement is always compatible with another.
ast.Return: lambda node: "(easy)",
# Calling these never compatible is wrong. Calling them
# always compatible will give lots of false positives.
ast.Expr: lambda node: "(maybe)",
# These ones are never compatible, so return some
# object that's never equal to anything.
ast.Import: lambda node: float("nan"),
ast.ImportFrom: lambda node: float("nan"),
ast.Pass: lambda node: float("nan"),
ast.Raise: lambda node: float("nan"),
ast.If: lambda node: float("nan"),
}

vs (my proposal):

with key from ast:
if Assign:
return ' '.join(dump(t) for t in node.targets)
elif AugAssign:

# Same target and same operator.

return dump(node.target) + dump(node.op) + "="
elif Return:

# A return statement is always compatible with another.

return "(easy)"
elif Expr:

# Calling these never compatible is wrong. Calling them
# always compatible will give lots of false positives.

return "(maybe)"
else:

# These ones are never compatible, so return some
# object that's never equal to anything.

return float("nan")

Which do *you* find more readable?


Marko

[Mark H. Harris]

On Friday, February 28, 2014 1:20:52 PM UTC-6, Marko Rauhamaa wrote:

>
> Which do *you* find more readable?
>

Yep, my point exactly. nice illustration.