anything like C++ references?
Tom Plunket
void change(int& i)
{
i++;
}
Is there any way to do references like this in Python? It'd be
like this, though clearly this doesn't change the thing I want it
to change:
def change(val):
val += 1
v = 0
change(v)
# v == 1 now.
thanks.
I can see that maybe if I passed in a list of one integer I could
change that, but that seems circuitous to me.
-tom!
"Martin v. Löwis"
> Is there any way to do references like this in Python?
No.
Martin
Ian Bicking
> > void change(int& i)
> > {
> > i++;
> > }
>
> The idiomatic way to write this code in python would be:
>
> def change(val):
> return val + 1
To be more specific, you would achieve the same effect with:
def change(val):
return val + 1
i = change(i)
As opposed to the C++ where you'd do:
change(i) // no assignment needed
There is no direct Python equivalent to the C++ function, for all sorts
of reasons (most of them very deliberate).
Is there someplace in particular to reference people who need to learn
about what Python variables are (that they are bindings, etc)? This
question comes up in many forms all the time, particularly from people
with a C background. Or rather, there are many questions all of which
are answered by that explanation (so we can't expect people to stop
asking, but maybe we can make the answers easier and clearer). Or maybe
a general Python-for-C-programmers tutorial...
Ian
Brian Quinlan
> {
> i++;
> }
The idiomatic way to write this code in python would be:
def change(val):
return val + 1
If you don't want to write it this way for some reason, you will have to
explain why.
Cheers,
Brian
Erik Max Francis
> Is there any way to do references like this in Python? It'd be
> like this, though clearly this doesn't change the thing I want it
> to change:
Hi, Tom.
There's no builtin support for this in Python, but you can do it
yourself easily enough. Pass in a mutable container, and then twiddle
the object you want inside it.
>>> def change(val):
... val[0] += 1
...
>>> v = [0]
>>> change(v)
>>> v
[1]
I'm a strong believer in writing self-documenting code, so in a case
like this I write a (trivial) class that makes my intent crystal clear:
>>> class Container:
... def __init__(self, value=None): self.value = value
... def get(self): return self.value
...
>>> class Container:
... def __init__(self, value=None): self.value = value
... def get(self): return self.value
... def set(self, value): self.value = value
...
>>> def change(container):
... container.set(container.get() + 1)
...
>>>
>>> c = Container(0)
>>> change(c)
>>> c.get()
1
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ Winners are men who have dedicated their whole lives to winning.
\__/ Woody Hayes
Dave Brueck
> On Sat, 2003-07-12 at 17:45, Brian Quinlan wrote:
> > > {
> > > i++;
> > > }
> >
> > The idiomatic way to write this code in python would be:
> >
> > def change(val):
> > return val + 1
>
>
> return val + 1
>
> As opposed to the C++ where you'd do:
>
> change(i) // no assignment needed
>
>
> There is no direct Python equivalent to the C++ function, for all sorts
> of reasons (most of them very deliberate).
Your response above (that the normal thing to do in Python is just return the
modified value) made me wonder if most people are asking about
pass-by-reference not because they want pass-by-reference, but because in C
it's generally a nuisance to return stuff from functions, especially multiple
values, so you end up learning about pointers and/or pass-by-reference.
IOW, if you could erase the influence of previous languages would this FAQ
become "how can I return multiple things from a function" more often than it
would become "how can I modify an object from inside a function"?
-Dave
Tom Plunket
> To be more specific, you would achieve the same effect with:
>
> def change(val):
> return val + 1
> i = change(i)
Yeah, thanks. I don't recall off-hand why it is that I wanted
this functionality, but something came up a few weeks ago that
seemed to beg for it. This was the solution that I used, but it
was non-optimal for some reason. The question came to mind
because of that global question posted just previously, to which
I made a good response if I may say so myself. :)
> There is no direct Python equivalent to the C++ function, for all
> sorts of reasons (most of them very deliberate).
Ok. I'd love to see the reasons for it. To me it just seems
like there's a somewhat arbitrary line between immutables and
mutables.
> Is there someplace in particular to reference people who need to
> learn about what Python variables are (that they are bindings,
> etc)?
Yeah, this is actually less a stumbling block, and more a desire
to want to bind to another variable instead of that variable's
value. I mean, if I do this:
a = [ ]
b = a
a.append(4)
print b
[ 4 ]
This seems inconsistent to me, although I understand that the two
variables are just binding to the same list. No different than
binding to the same tuple or integer or string, it's just that
changing the value of a tuple, an integer, or a string means
rebinding your variable to a different object.
I suppose one "solution" would be a class that wraps the value,
so the value can change but the object stays the same. Similar
is using a list, but these two options make it a little less
convenient, and one must exercise extreme caution lest one
accidentally reassign as an int instead of changing the internals
of the wrapper object.
> This question comes up in many forms all the time, particularly
> from people with a C background. Or rather, there are many
> questions all of which are answered by that explanation...
IMHO the variable binding discussion doesn't answer the question,
it just provokes more questions. :)
thanks to all who responded.
-tom!
Tom Plunket
> Your response above (that the normal thing to do in Python is
> just return the modified value) made me wonder if most people are
> asking about pass-by-reference not because they want pass-by-
> reference, but because in C it's generally a nuisance to return
> stuff from functions, especially multiple values, so you end up
> learning about pointers and/or pass-by-reference.
In C++ it's trivial to return multiple values and pass-by-
reference is typically (at least, as far as I've seen) used to
pass immutables anyway, but sometimes it just seems like the
right tool for the job.
> IOW, if you could erase the influence of previous languages would
> this FAQ become "how can I return multiple things from a
> function" more often than it would become "how can I modify an
> object from inside a function"?
That's a good idea, although I would have to say that multiple
return values typically means that your function is doing too
many things. ;)
-tom!
Erik Max Francis
> IMHO the variable binding discussion doesn't answer the question,
> it just provokes more questions. :)
But the answer to those questions is usually something ending with,
"Look, Python just doesn't handle variables the same way C++ does." :-)
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ It seems like Karma's making his rounds
\__/ Kina
David McNab
breath, then came out with:
> I want to do something along the lines of the following C++ code:
>
> void change(int& i)
> {
> i++;
> }
> Is there any way to do references like this in Python?
<snip>
In Python, basic types like strings and numbers are a weird exception to
the 'everything is an object' rule.
When you pass any other object in a function, the function gets a ref to
that object.
But when you pass a string or numeric object, the whole thing (not a ref)
gets passed.
Your best bet would be to create a wrapper class for numbers, store the
actual number in an attribute, and provide all the methods in Section
3.3.6 of the Python Reference Manual (__add__, __sub__ etc) to work on the
value attribute.
That way, you can use your numeric object in calculations, and when you
pass it to functions (like your 'change()' above), things will work as
expected. In fact, in most situations, it will look/feel/smell just like
a number.
But watch out for trying to assign to it, or using '+=' type operators -
they will replace your object with a plain number.
I attach below a sample class declaration for a numeric type which is
passable by reference (no need to do this for string, since you can just
use the UserString module).
Cheers
David
class myint:
def __init__(self, value):
self.value = value
def __getattr__(self, attr):
if attr in ['__repr__', '__add__', '__repr__', '__add__',
'__sub__', '__mul__', '__floordiv__', '__mod__',
'__divmod__', '__pow__', '__lshift__',
'__rshift__', '__and__', '__xor__', '__or__',
'__div__', '__truediv__', '__radd__', '__rsub__',
'__rmul__', '__rdiv__', '__rtruediv__',
'__rfloordiv__', '__rmod__', '__rdivmod__',
'__rpow__', '__rlshift__', '__rrshift__',
'__rand__', '__rxor__', '__ror__',
'__neg__',
'__pos__', '__abs__', '__invert__',
'__complex__', '__int__', '__long__',
'__float__', '__oct__', '__hex__', '__coerce__']:
return getattr(self.value, attr)
def __iadd__(self, other):
self.value += other
return self.value
def __isub__(self, other): self.value -= other
def __imul__(self, other): self.value *= other
def __idiv__(self, other): self.value /= other
def __itruediv__(self, other):
self.value = self.value.__itruediv__(other)
def __ifloordiv__(self, other): self.value = self.value.__itruediv__(other)
def __imod__(self, other): self.value = self.value.__imod__(other)
def __ipow__(self, other): self.value = self.value.__ipow__(other)
def __ilshift__(self, other): self.value = self.value.__ilshift__(other)
def __irshift__(self, other): self.value = self.value.__irshift__(other)
def __iand__(self, other): self.value = self.value.__iand__(other)
def __ixor__(self, other): self.value = self.value.__ixor__(other)
def __ior__(self, other): self.value = self.value.__ior__(other)
Dave Brueck
> Dave Brueck wrote:
> > Your response above (that the normal thing to do in Python is
> > just return the modified value) made me wonder if most people are
> > asking about pass-by-reference not because they want pass-by-
> > reference, but because in C it's generally a nuisance to return
> > stuff from functions, especially multiple values, so you end up
> > learning about pointers and/or pass-by-reference.
>
> In C++ it's trivial to return multiple values
Trivial but non-intuitive until you convince yourself "that's the way it is",
perhaps. :) What I mean is this: when you're just starting out you learn that
functions can return values like this:
int foo(int a, int b)
{
return a + b;
}
But the method for returning more than one thing is not a simple progression
from this pattern. Instead you learn and shift to a *completely* different
mechanism. In Python, however, you *can* continue along the original route as
well:
def foo(a, b, c):
return a+b, b+c
> > IOW, if you could erase the influence of previous languages would
> > this FAQ become "how can I return multiple things from a
> > function" more often than it would become "how can I modify an
> > object from inside a function"?
>
> That's a good idea, although I would have to say that multiple
> return values typically means that your function is doing too
> many things. ;)
Far from it:
lastName, firstName = line.split('\t')
(there are many, many, many such examples). This is also an example of what I
mean by "in C it's genreally a nuisance to return multiple values":
void split(char *pLine, char **ppLastName, char **ppFirstName) {}
Sadly, it's not uncommon to see one-off structures defined whose sole purpose
in life is to act as a go-between for functions. Ugh.
-Dave
Stephen Horne
wrote:
>Tom Plunket wrote:
>
>> IMHO the variable binding discussion doesn't answer the question,
>> it just provokes more questions. :)
>
>But the answer to those questions is usually something ending with,
>"Look, Python just doesn't handle variables the same way C++ does." :-)
And my traditional answer to that is... "yes, and the way that Python
handles variables is definitely a wart."
One of the few things I hate about Python is that mutable objects are
implicitly shared using a reference system whereas immutable objects
are not. It is unnecessarily confusing and error prone.
A system where all objects are copied unless a reference type is
explicitly applied (and with an assignment operators recognising
references on the LHS by default and assigns to the object rather than
the reference, though with an extra one which overwrites the
reference) seems much more logical to me. It even seems 'Pythonic' to
me. But it is much closer to C++ references than current Python
system.
Stephen Horne
<da...@pythonapocrypha.com> wrote:
>On Sunday 13 July 2003 01:41 am, Tom Plunket wrote:
>> In C++ it's trivial to return multiple values
>
>Trivial but non-intuitive until you convince yourself "that's the way it is",
>perhaps. :) What I mean is this: when you're just starting out you learn that
>functions can return values like this:
>
>int foo(int a, int b)
>{
> return a + b;
>}
>
>But the method for returning more than one thing is not a simple progression
>from this pattern. Instead you learn and shift to a *completely* different
>mechanism. In Python, however, you *can* continue along the original route as
>well:
>
>def foo(a, b, c):
> return a+b, b+c
You are not directly returning multiple values - you are returning a
tuple. You can return containers in C++ too. The only difference is
that Python makes working with tuples *much* easier - but its no more
a 'simple progression' than in C++. You still need to know about
tuples as well as returning values.
>> > IOW, if you could erase the influence of previous languages would
>> > this FAQ become "how can I return multiple things from a
>> > function" more often than it would become "how can I modify an
>> > object from inside a function"?
There's more to it than this.
Take the Pascal family of languages. These have an explicit
distinction between functions and procedures which is hidden in C and
Python.
In my view, the distinction between a procedure (an imperative tool
which may modify parameters in place) and a function (which returns a
value but - normally, at least - leaves its parameters unmodified) is
a useful one. Writing code to respect these conventions, even when the
language doesn't explicitly support it, has considerable benefits -
evaluation order in expressions becomes much less of an issue, for
instance, effectively eliminating at least one source of subtle
hard-to-trace errors.
Which suggests a new 'procedure' syntax for Python, either
automatically treating all parameters as references or providing a
syntax equivalent to the Pascal 'var'.
Oren Tirosh
> On Sat, 12 Jul 2003 13:53:35 -0700, Tom Plunket paused, took a deep
> breath, then came out with:
>
> > I want to do something along the lines of the following C++ code:
> >
> > void change(int& i)
> > {
> > i++;
> > }
>
> > Is there any way to do references like this in Python?
> <snip>
>
> In Python, basic types like strings and numbers are a weird exception to
> the 'everything is an object' rule.
>
> When you pass any other object in a function, the function gets a ref to
> that object.
>
> But when you pass a string or numeric object, the whole thing (not a ref)
> gets passed.
Numbers and strings are objects just like everything else in Python and
are passed by reference. The only difference is that they are *immutable*
objects so their value cannot be changed through that reference in a way
that will be visible to others who reference the same object. Under most
circumstances a reference to an immutable object is nearly indistingishable
from a value. But the id() builtin function or the 'is' operator quickly
reveal the difference.
There is no weird exception to any rule here. Some objects are mutable,
some are not. That's all.
Oren
Roy Smith
> One of the few things I hate about Python is that mutable objects are
> implicitly shared using a reference system whereas immutable objects
> are not. It is unnecessarily confusing and error prone.
There's a bit of confusion in the above regarding mutability vs. call by
reference (they're orthogonal ideas), but I think I know what you were
trying to say.
In ancient versions of Fortran, even constants (i.e. immutables) were
pass by reference. You could write something like this (forgive me if
I've forgotten some of the syntax details):
subroutine x (i)
i = 42
end
subroutine main
x (5)
write (6, 100) 5
100 format (1i6)
end
and it would print 42. There was a single instance of the constant 5
for main, and it was passed by reference to x(), which in turn changed
its value to 42. I don't remember if Fortan 66 did this, or if that
only happened in even older versions. Talk about confusing and error
prone!
The Python version of that would be:
def mutateString (s):
s = "some other string"
s ("foo")
print "foo"
If strings were pass by reference, the above would print "some other
string". Unless, of course, you got away from the idea that multiple
instances of a string constant with the same value are really the same
object, in which case:
x = "foo"
x is "foo"
would print 0 instead of 1 like it does now.
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>
>One of the few things I hate about Python is that mutable objects are
>implicitly shared using a reference system whereas immutable objects
>are not.
Well, that's incorrect. *ALL* Python objects are implicitly shared with
bindings. The difference is whether updating the value referenced by a
target requires *re*binding the target or simply updating an object.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"Not everything in life has a clue in front of it...." --JMS
Stephen Horne
>In article <iqm2hvs77lhe1m71v...@4ax.com>,
>Stephen Horne <intent...@blank.co.uk> wrote:
>>
>>One of the few things I hate about Python is that mutable objects are
>>implicitly shared using a reference system whereas immutable objects
>>are not.
>
>Well, that's incorrect. *ALL* Python objects are implicitly shared with
>bindings. The difference is whether updating the value referenced by a
>target requires *re*binding the target or simply updating an object.
Fine - nit-pick.
All you have proven is that it is the distinction between types that
get re-bound and those that don't (rather than the use of references)
that is unnecessarily confusing and error prone.
The wart remains, even if my description was wrong. And even that is a
dubious claim.
A Python user is interested in how an object behaves - not how it is
internally implemented in the interpreter. Immutable objects don't
behave as references - the internal use of references for immutable
objects is basically a lazy copying optimisation and, apart from
performace and a couple of other technicalities (e.g. the 'is'
operator), has no relevance. Certainly it has no relevance to the
point I was making.
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>On 13 Jul 2003 10:37:01 -0400, aa...@pythoncraft.com (Aahz) wrote:
>>In article <iqm2hvs77lhe1m71v...@4ax.com>,
>>Stephen Horne <intent...@blank.co.uk> wrote:
>>>
>>>One of the few things I hate about Python is that mutable objects are
>>>implicitly shared using a reference system whereas immutable objects
>>>are not.
>>
>>Well, that's incorrect. *ALL* Python objects are implicitly shared with
>>bindings. The difference is whether updating the value referenced by a
>>target requires *re*binding the target or simply updating an object.
>
>Fine - nit-pick.
Believe it or not, it is *not* a nit-pick, it goes right to the core of
Python's design and philosophy.
>All you have proven is that it is the distinction between types that
>get re-bound and those that don't (rather than the use of references)
>that is unnecessarily confusing and error prone.
Only for people who insist on thinking about it through the lens of some
other way to do it.
>A Python user is interested in how an object behaves - not how it is
>internally implemented in the interpreter. Immutable objects don't
>behave as references - the internal use of references for immutable
>objects is basically a lazy copying optimisation and, apart from
>performace and a couple of other technicalities (e.g. the 'is'
>operator), has no relevance. Certainly it has no relevance to the
>point I was making.
That's precisely where you are wrong. Python's object model is
extremely strict and simple and orthogonal. Everything works exactly
the same way. Whether one can mutate a specific object is simply an
attribute of that object, rather than requiring a different syntax.
Trying to focus on the mutable/immutable distinction is what causes the
mental blowup -- keep your eye on the objects and bindings and you're
fine.
Noah
As others have explained, you just return the value.
It's just a different point of view. Instead of
change (v)
you have:
v = change (v)
I was a C++ programmer in a former life and I remember being
annoyed at the lack of references in Python -- I mean, why would
something so obvious and cool as a reference variable be excluded, right?
Where people usually find themselves wanting references is
when they need to modify more than one value at once.
For example, consider a function to rotate a 3D point through
two angles, where x,y,z get modified by the rotate function:
rotate (x,y,z, theta, phi)
Here you just have to remember that Python can return more than one
value at once as a tuple. So you should expect the code to look like this:
x,y,z = rotate (x,y,z, theta, phi)
Your rotate() function would return a simple tuple as:
return (x,y,z)
After I unlearned the C++ idiom I decided I liked the Python idiom better.
It's a teensy bit more verbose, but it is far more clear as to what
is going on. Your inputs and output are clearly defined. With C++ references
you have to look up the function definition to know which variables in modified.
Yours,
Noah
Aahz
Tom Plunket <to...@fancy.org> wrote:
>
>I want to do something along the lines of the following C++ code:
>
>void change(int& i)
>{
> i++;
>}
class Counter:
def __init__(self, start=0):
self.counter = start
def inc(self, increment=1):
self.counter += increment
def change(counter):
counter.inc()
There's no need to get into all the special methods required to
implement a number type, and it's a Bad Idea to use a random container
object. Just pass around a mutable class instance, and your code
becomes simpler and clearer.
Stephen Horne
>Whether one can mutate a specific object is simply an
>attribute of that object, rather than requiring a different syntax.
>Trying to focus on the mutable/immutable distinction is what causes the
>mental blowup -- keep your eye on the objects and bindings and you're
>fine.
That's exactly it - you have to focus on whether an object is mutable
or immutable for that exact reason.
While I admit I'm not sure, I believe that in early versions of Python
immutable values literally were copied. Mutable types were made
mutable because of efficiency concerns about copying overheads that
would occur otherwise. If an object is large enough that it is
worthwhile modifying items within it in-place then it tends to be
worthwhile trying to avoid copying overheads, even though these two
aspects of 'mutability' are actually quite distinct in principle.
Actually, I'm convinced there are some flaws in Guidos own logic
*unless* this is the case. Take for instance this...
http://www.python.org/search/hypermail/python-1992/0292.html
"""
<A> They serve different purposes. Lists can get quite long, they are
generally built incrementally, and therefore have to be mutable.
Tuples on the other hand are generally short and created at once.
<Q> Then why can't tuples be mutable and serve both purposes?
<A> Imagine a graphics class that stores coordinates of graphical
objects as tuples. It may also store rectangles as a tuple of points,
etc. If tuples were mutable, the class would have to store copies of
all points and rectangles it receives, since otherwise a caller who
creates a point variable, passes its value to the graphics class, and
later changes the point for its own use (e.g., to create another
graphical object with slightly different coordinates) might violate
the internal consistency of the graphics class. Note that most
callers woouldn't modify the points, but the graphics class has no way
to tell, so it has to make the copies anyway. (And now imaging the
software is actually layered in such a way that coordinates are passed
down several levels deep...)
"""
So what?
As things are, the points could be stored using a list of tuples, with
each tuple representing a point. When you store a new point in the
list, you don't immediately make a copy (so if the supplier of the
point immediately discards it, no copying is needed at all except for
pointers) so it is efficient - yet you are still guarenteed that
external processing will not mutate the value you hold in your list.
Fine.
But that can be achieved equally well by applying a copy-on-write
system to *all* types, without a distinction between mutable and
immutable for this purpose.
Equally, you could use the same rationale for storing items which
Python classes as mutable instead of these point tuples. If you have
an object which stores class instances or dictionaries or lists for
you, you need to store copies as - just as Guido recognised for the
tuple case - your object doesn't know for sure what the caller is
going to do with those objects after passing them in as parameters.
This makes no sense to me *unless* there were other reasons for the
mutable/immutable distinction. If lists had copy-on-write semantics,
they could still be mutable (ie support in-place modification) yet
there would be no need to make early copies. Tuples would be redundant
as their only distinctive feature would be their lack of support for
in-place modifications. It only makes sense if the copy-on-write for
immutable values hadn't been implemented when the distinction was
created - if immutables did not use references, but were always copied
immediately on assignment - so that the referencing mechanism need
never worry about copy-on-write at all.
But it is worse than just being an arbitrary and pointless
distinction...
Back when PEP238 was being 'debated', a major argument in favor of the
change was that there is no legitimate case where a function would
need to either do 'true' division or integer division depending on the
argument type. The type of division needed depends on the purpose of
the function, and behaviour shouldn't change just because someone
passes an integer into a function that expects a real or visa versa.
Apply the same logic to mutable and immutable types.
Tell me one case where it is sensible for a function to behave such
that whether the caller sees a change in a variable it passed as its
argument should depend on the type.
Tell me one case where an object storing values should care about
callers mutating values it holds *only* for certain types.
I doubt you can.
This is an arbitrary distinction created for historic reasons that
have long since vanished (apart, of course, from backward
compatibility and psychological inertia), given that copy-on-write is
implemented for immutable objects anyway. And the fact that immutable
values do an excellent job of implementing the common semantics of
most imperitive languages (i.e. assignment sets the LHS to something
that can be treated as a distinct copy) whereas mutable types behave
in a fundamentally different way is quite simply a wart -
inconsistent, confusing and error-prone behaviour.
Terry Reedy
"David McNab" <postm...@127.0.0.1> wrote in message
news:pan.2003.07.13....@127.0.0.1...
> In Python, basic types like strings and numbers are a weird
exception to
> the 'everything is an object' rule.
In Java perhaps, but NOT in Python. This is WRONG, WRONG, WRONG.
Please do not muddy the waters like this.
In Python, most builtin types are immutable. The two mutable builtin
types are lists and dicts. Instances of user-defined classes are also
mutable by default.
>When you pass any other object in a function,
>But when you pass a string or numeric object, [incorrect statement
clipped]
In Python, everything is 'passed' to functions the same way: first,
the argument expressions are evaluated one at a time, left to right.
In the typical case (number of argument expressions == number of
positional parameters), the resulting objects are then bound to the
corresponding parameter names, left to right, in the local namespace
of the function. (Or, if you prefer, the names are bound ...) The
type of the argument objects is not looked at.
How a particular interpreter performs name binding is its own
business, as long as the specified semantics are implemented. What
CPython does may or may not be the same as Jython or any other actual
or potential computer implementation. What any computer does is
probably significantly different from what human interpreters do.
Terry J. Reedy
Donn Cave
| On 13 Jul 2003 12:19:08 -0400, aa...@pythoncraft.com (Aahz) wrote:
|
|> Whether one can mutate a specific object is simply an
|> attribute of that object, rather than requiring a different syntax.
|> Trying to focus on the mutable/immutable distinction is what causes the
|> mental blowup -- keep your eye on the objects and bindings and you're
|> fine.
|
| That's exactly it - you have to focus on whether an object is mutable
| or immutable for that exact reason.
|
| While I admit I'm not sure, I believe that in early versions of Python
| immutable values literally were copied. Mutable types were made
| mutable because of efficiency concerns about copying overheads that
| would occur otherwise. If an object is large enough that it is
| worthwhile modifying items within it in-place then it tends to be
| worthwhile trying to avoid copying overheads, even though these two
| aspects of 'mutability' are actually quite distinct in principle.
Sure, mutable values literally are copied today. If you have a tuple
of time.localtime(x), for example, and you want that value but with
tm_mday incremented, then you must copy the tuple to do that (except
for tm_mday, for which you substitute your own value.) It is indeed
inefficient - OK for this purpose but not for large sequences, and
there we tend to prefer a mutable list.
That's the efficiency argument. Assigment never copied values, in
any version of Python that anyone cared about. We just don't want
big immutable arrays that have to be copied just to change a single
value. There isn't a tuple analogue for the dictionary because it
wouldn't be so generally useful (the recently implemented struct
tuple would have been nice to have earlier, though.)
He's quite right, the notion of mutability is a red herring. It
isn't that no one cares - the propagation of state changes in a
program is critical to the operation of the program, and to reasoning
about how the program will operate.
It's just that when you're trying to come to grips with how assignment
and argument passing works in Python, you don't want to have people
start explaining things in these terms (``see, this object is mutable
and that one isn't''), because you will understandably but incorrectly
apply that to assignment and argument passing. Mutability in Python
has no effect on assignment and argument passing, it is just an attribute
of the passed or assigned object.
Donn Cave, do...@drizzle.com
Stephen Horne
wrote:
>How a particular interpreter performs name binding is its own
>business, as long as the specified semantics are implemented.
You could use the same argument to rationalise a language which used
the + operator to express subtraction.
The fact is that 'assignment' has a common meaning separate from the
choice of programming language, and that the way Python handles name
binding means that meaning is not respected by Python for mutable
objects.
>What any computer does is
>probably significantly different from what human interpreters do.
And using references with a copy-on-write or lazy-assignment system is
different to immediate copying of values, but that is irrelevent as
both systems respect the common meaning of assignment.
Python, however, does not respect this meaning. It uses an arbitrary
distinction based on whether or not the object has internals that can
be usefully in-place modified, and uses that distinction to vary the
effect of assignment in an arbitrary, inconsistent and error-prone
way. The need for a distinct tuple type is a symptom of the arbitrary
nature of this.
I've already posted too much on this, so this is going to be the end
to it. But Pythons semantics with regard to mutable and immutable
objects are simply bizarre. A historic justification may be plausible
and backward compatibility is then an important issue, but I really
cannot accept claims that Python is doing the right thing here.
Ian Bicking
> On 13 Jul 2003 12:19:08 -0400, aa...@pythoncraft.com (Aahz) wrote:
>
> >Whether one can mutate a specific object is simply an
> >attribute of that object, rather than requiring a different syntax.
> >Trying to focus on the mutable/immutable distinction is what causes the
> >mental blowup -- keep your eye on the objects and bindings and you're
> >fine.
>
> That's exactly it - you have to focus on whether an object is mutable
> or immutable for that exact reason.
No you don't! Mutable and immutable objects act the same with respect
to assignment. However, because you cannot change a immutable object in
place, to get a different value you must rebind. You must *use* mutable
and immutable objects differently -- but that is not surprising, because
they are obviously very different objects! You have to use integers and
lists differently in part because one is mutable and the other isn't --
but mostly because one is a number and the other is a list. Different
objects are different!
Python is not novel in the way it deals with variables. Scheme and
Smalltalk, for instance, act exactly the same, as do many other
dynamically typed languages (though there are different opinions on
whether strings should be mutable -- but it's agreed there has to be
some immutable string-like type, e.g. symbol). The reason you are
getting this reaction is that anyone that comes from those backgrounds
thinks you are crazy, as does anyone who has embraced the Python model.
This isn't a funny little feature, this is the way all strong,
dynamically typed languages work.
> While I admit I'm not sure, I believe that in early versions of Python
> immutable values literally were copied. Mutable types were made
> mutable because of efficiency concerns about copying overheads that
> would occur otherwise. If an object is large enough that it is
> worthwhile modifying items within it in-place then it tends to be
> worthwhile trying to avoid copying overheads, even though these two
> aspects of 'mutability' are actually quite distinct in principle.
The problem you have is you are still thinking of variables as slots,
which is not correct. Variables in Python are bindings. Assignment
never copies anything, or creates anything except for changing the
variable to point to a different address location. *Every* Python
assignment (a=b) is like the C assignment (a=&b).
> Tell me one case where it is sensible for a function to behave such
> that whether the caller sees a change in a variable it passed as its
> argument should depend on the type.
Generally a function takes either immutable values (e.g., ints and
floats) or mutable values for a certain argument.
However, there is a class of operations which are generally operate in
an immutable manner, that is, create copies of the objects instead of
changing them in place. So even though lists are mutable, list
concatenation does not mutate, and in general adding two things (with +)
will not mutate either object. Immutable values, by design, do not have
the same methods and operations as the mutable counterparts (or at least
the mutating methods of those mutable objects). *That* would be a
design bug.
> Tell me one case where an object storing values should care about
> callers mutating values it holds *only* for certain types.
Objects don't *store* values, they *refer* to values. You are still
thinking like you're in C (or C++). This is why you are having a
problem.
Ian
Ian Bicking
> You are not directly returning multiple values - you are returning a
> tuple. You can return containers in C++ too. The only difference is
> that Python makes working with tuples *much* easier
Of course, anything you can do in Python you can do in any other
language. The *whole point* is that Python is so much easier. Don't
worry so much about every equivalent feature or about translating every
C++ idiom to Python. Just write your programs -- they'll be different
from C++, but that's okay.
Ian
Erik Max Francis
> In Python, basic types like strings and numbers are a weird exception to
> the 'everything is an object' rule.
>
> When you pass any other object in a function, the function gets a ref to
> that object.
>
> But when you pass a string or numeric object, the whole thing (not a ref)
> gets passed.
This is completely incorrect. On the contrary, builtin types are
handled exactly uniformly in Python. The only difference here is that
the builtin types that you've listed, along with some others, are
immutable, so you cannot change them if you have a reference to them.
Contrast this to, say, Java, where there really is a solid distinction
between builtin fundamental types, like int and long, and actual object
types, derived from java.lang.Object, which are all passed by reference.
Python doesn't make this distinction.
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ But who shall dwell in these worlds if they be inhabited?
\__/ Johannes Kepler
Tim Peters
> ...
> The fact is that 'assignment' has a common meaning separate from the
> choice of programming language,
Well, that's not a fact, although there may be much commonality among the
(necessarily few, if you believe this) programming languages you've used.
Python's assignment semantics are in fact common among languages
higher-level than those of the Fortran/COBOL/Algol/C ilk. For examples,
Lisp, SNOBOL4, Scheme, Smalltalk and Icon are (very) much like Python in
this respect. So is Java in the end, although the underlying machinery is
more complicated there.
> and that the way Python handles name binding means that meaning is not
> respected by Python for mutable objects.
Python's assignment semantics don't vary according to object type.
> ...
> Python, however, does not respect this meaning.
This doesn't say more than that Python works differently than the languages
you've used. It really isn't a question of respect <wink>.
> It uses an arbitrary distinction based on whether or not the object
> has internals that can be usefully in-place modified, and uses that
> distinction to vary the effect of assignment in an arbitrary,
> inconsistent and error-prone way.
It's deliberate and thoroughly consistent. Any assignment semantics are
error-prone in some contexts, though (Python's included -- side effects are
like that). The simplicity and absolute consistency of Python-style
assignment semantics are probably easiest for first-time programmers to pick
up, and certainly dead easy for people coming from Lisp (etc) to pick up --
people coming from C (etc) often struggle with inappropriate expectations at
first.
> The need for a distinct tuple type is a symptom of the arbitrary
> nature of this.
Python's assignment semantics have nothing to do with mutability versus
immutability. The latter is a type-dependent distinction; assignment
semantics in Python don't depend on object type in any way.
> ...
> I've already posted too much on this, so this is going to be the end
> to it. But Pythons semantics with regard to mutable and immutable
> objects are simply bizarre.
Hmm. Mutable objects can be mutated, and immutable objects cannot be
mutated. That doesn't seem bizarre.
> A historic justification may be plausible and backward compatibility
> is then an important issue, but I really cannot accept claims that
> Python is doing the right thing here.
Well, I can assure you Guido wouldn't change anything about Python's
assignment semantics if were able to do it all over from scratch. That's a
puzzle for you to work out, then: is Guido unique, or are all Dutch people
that stupid <wink>?
Ian Bicking
> The fact is that 'assignment' has a common meaning separate from the
> choice of programming language,
This just isn't true. The C++ assignment operator is not at all like
the Python assignment statement. Python variables are not like C++
variables, no surprise assignment is different too. If you used
languages outside of C++ and its like (e.g., Pascal), you would find
Python's behavior common.
(Admittedly, some confusion may occur because these very different
operations use the same syntax:
x = 10
x[0] = 10
obj.x = 10
The second and third are entirely different from the first.)
Ian
Stephen Horne
wrote:
In computer science, a variable is a named binding to a value.
Operations on that variable may rebind it to a different value, but
values don't change behind your back. A pointer is, in effect, a value
which indirectly refers to another (possibly anonymous) variable. A
pointer is something you specifically request, and by doing so you
allow that the 'variable' being indirectly referenced may be modified
by something else that has no direct access to your pointer value (via
your named variable or via copys or pointers-to your pointer held
elsewhere).
Python should respect that.
If you claim that it does, then the values to which we currently
associate the type name 'list', 'dictionary' and 'class instance' are
badly named. They should be called 'pointer to list', 'pointer to
dictionary' and 'pointer to class instance'. And if you want to call
those references and make the dereferencing implicit, fine. Equally,
if you want to implement variable binding using references and using
copy-on-write to implement lazy copying (which does not violate the
standard computer-theory semantics of the binding to values - only the
binding to the implementation of values ie memory locations) then
equally fine.
However, even if the names included the 'pointer to' prefix so that
they actually described the real behaviours of the values, this still
raises two important questions...
1. Why are the values of mutable objects always forced to be accessed
via a pointer (or reference or whatever)?
Note that I am referring to a computer science, semantic
pointer/reference/whatever - not to any pointers or references that
may be used in the implementation of the binding of variables to
values. The use, behind the scenes, of lazy copying as an optimisation
is irrelevant to the computer science principles.
2. Why is there no way to reference an immutable object via a
pointer, other than stuffing it into a mutable object designed for
some purpose other than simple pointer behaviour?
The truth is that this system is arbitrary, and that therefore this
excuse is invalid.
>The problem you have is you are still thinking of variables as slots,
>which is not correct. Variables in Python are bindings. Assignment
>never copies anything, or creates anything except for changing the
>variable to point to a different address location. *Every* Python
>assignment (a=b) is like the C assignment (a=&b).
The problem is that you are confusing implementation with semantics.
The meanings of 'variable', 'value', 'assignment' etc are defined by
computer science. Whether you arbitrarily change the meaning of
'assignment' or whether you arbitrarily change the meaning of
'variable', it amounts to the same thing.
>
>> Tell me one case where it is sensible for a function to behave such
>> that whether the caller sees a change in a variable it passed as its
>> argument should depend on the type.
>
>Generally a function takes either immutable values (e.g., ints and
>floats) or mutable values for a certain argument.
>
>However, there is a class of operations which are generally operate in
>an immutable manner, that is, create copies of the objects instead of
>changing them in place. So even though lists are mutable, list
>concatenation does not mutate, and in general adding two things (with +)
>will not mutate either object. Immutable values, by design, do not have
>the same methods and operations as the mutable counterparts (or at least
>the mutating methods of those mutable objects). *That* would be a
>design bug.
You miss my point. Your argument would equally apply to the need for
an integer-specific division operator, for instance. Its all about how
one function reacts in response to varying input. If you pass a
mutable value to a function which expects an immutable one, you get an
error. If you pass an immutable value to a function which expects a
mutable one, you get an error. There is no good reason for a function
to react to the distinction as evidenced by your own observation that
what actually happens (and what should happen) is that you have two
distinct variants of the function.
>> Tell me one case where an object storing values should care about
>> callers mutating values it holds *only* for certain types.
>
>Objects don't *store* values, they *refer* to values. You are still
>thinking like you're in C (or C++). This is why you are having a
>problem.
No it is not. I'm thinking in terms of computer science, in which
terms like 'variable', 'value' and 'assignment' are abstract concepts
independent of the way in which they are implemented in a programming
language.
One way or the other, Python is currently choosing not to respect the
computer science definition of those terms. It may have historic and
backward-compatability reasons, but that does not change the facts.
This deviation from computer science definitions, whatever the excuse,
is arbitrary, confusing and error prone. That is my problem.
Terry Reedy
"Stephen Horne" <intent...@blank.co.uk> wrote in message
news:b8c3hvsesjoumiboo...@4ax.com...
> On Sun, 13 Jul 2003 15:07:23 -0400, "Terry Reedy" <tjr...@udel.edu>
> wrote:
>
> >How a particular interpreter performs name binding is its own
> >business, as long as the specified semantics are implemented.
>
> You could use the same argument to rationalise a language which used
> the + operator to express subtraction.
Bogus comparison with name binding. I am not aware that Python does
anything quite so backwards.
> The fact is that 'assignment' has a common meaning separate from the
> choice of programming language, and that the way Python handles name
> binding means that meaning is not respected by Python for mutable
> objects.
From what others have posted, Python is not the only language in which
'name=object' means "assign 'name' to that object". This is much
closer to the common idea of object having names than the idea that
names can only name value holders (blocks of linear memory) and not
values or objects themselves.
Terry J. Reedy
Stephen Horne
wrote:
>On Sun, 2003-07-13 at 14:39, Stephen Horne wrote:
>> The fact is that 'assignment' has a common meaning separate from the
>> choice of programming language,
>
>This just isn't true. The C++ assignment operator is not at all like
>the Python assignment statement. Python variables are not like C++
>variables, no surprise assignment is different too. If you used
>languages outside of C++ and its like (e.g., Pascal), you would find
>Python's behavior common.
Think again.
When I say "'assignment' has a common meaning separate from the choice
of programming language" I assumed you would get the hint. I'm not
referring to some specific other programming language, of which I have
used many - and not all of them imperative. I am referring to the
definitions in computer theory, which do not relate to any specific
programming language but actually apply to all of them, irrespective
of paradigm and everything else.
Of course, if you really believe that pointer/reference behaviour
should be arbitrarily tied to mutability then you can claim that I am
wrong, but you still can't claim the high ground as this is still an
arbitrary and bizarre thing to do. The ability to change part or all
of a value in-place has nothing to do with whether that value is
referenced using a pointer or whatever in computer theory - any link
between pointers/references and mutability should be related to the
implementation of the language - not the semantics.
So much for dropping out of the discussion, but I hate it when people
make false claims about my beliefs, making me out to be ignorant, when
it is *not* *me* who is missing the point.
Stephen Horne
wrote:
>> You could use the same argument to rationalise a language which used
>> the + operator to express subtraction.
>
>Bogus comparison with name binding. I am not aware that Python does
>anything quite so backwards.
Using an exaggeration as an analogy. Possibly. Though terms like
'assignment', 'variable' and 'value' seems pretty fundamental to me,
so the exaggeration is not that huge.
>> The fact is that 'assignment' has a common meaning separate from the
>> choice of programming language, and that the way Python handles name
>> binding means that meaning is not respected by Python for mutable
>> objects.
>
>From what others have posted, Python is not the only language in which
>'name=object' means "assign 'name' to that object". This is much
>closer to the common idea of object having names than the idea that
>names can only name value holders (blocks of linear memory) and not
>values or objects themselves.
Many languages have *explicit* ways to create what computer science
calls pointers. Sometimes they have implicit dereferencing. Sometimes
they come prepackaged in a 'pointer to' or 'reference to' style
object. But the fact that you are dealing with something that
semantically behaves as a pointer is always explicit in some way or
another.
Or rather, it should be. The fact that a few other languages are
broken in much the same way does not mean that Python is doing the
right thing. The right thing does not mean imitating Java or whatever
any more than it means imitating C++.
If you want a good example to imitate, though, look to Haskell. I
don't know how it binds variables to values, but I do know that in
general it can't a simple mapping from identifier to memory location.
Even so, I don't care. Why should I? The implementation of the binding
is irrelevant - I only care that the value bound to the variable is
the one that I specified and that it doesn't get changed behind my
back.
Stephen Horne
wrote:
>[Stephen Horne]
>> ...
>> The fact is that 'assignment' has a common meaning separate from the
>> choice of programming language,
>
>Well, that's not a fact, although there may be much commonality among the
>(necessarily few, if you believe this) programming languages you've used.
>Python's assignment semantics are in fact common among languages
>higher-level than those of the Fortran/COBOL/Algol/C ilk. For examples,
>Lisp, SNOBOL4, Scheme, Smalltalk and Icon are (very) much like Python in
>this respect. So is Java in the end, although the underlying machinery is
>more complicated there.
>
>> and that the way Python handles name binding means that meaning is not
>> respected by Python for mutable objects.
>
>Python's assignment semantics don't vary according to object type.
OK then. With reference to another of my posts...
1. Why are dictionarys called dictionaries (rather than references to
dictionarys or whatever)? Similar for lists and class instances.
2. Why are the values of mutable objects always forced to be accessed
via a pointer (or reference or whatever)?
Note - whether it makes sense to allow in-place changes to part or all
of a value is not the same as whether it makes sense to reference that
value indirectly.
3. Why is there no way to reference an immutable object via a
pointer, other than stuffing it into a mutable object designed for
some purpose other than simple pointer behaviour?
This excuse is, as I already said, invalid.
>Well, I can assure you Guido wouldn't change anything about Python's
>assignment semantics if were able to do it all over from scratch. That's a
>puzzle for you to work out, then: is Guido unique, or are all Dutch people
>that stupid <wink>?
Your words, not mine. I simply point out that no-one, BDFL or
otherwise, can be perfect.
Jimmy Retzlaff
>...
>Of course, if you really believe that pointer/reference behaviour
>should be arbitrarily tied to mutability then you can claim that I am
>wrong, but you still can't claim the high ground as this is still an
>arbitrary and bizarre thing to do. The ability to change part or all
>of a value in-place has nothing to do with whether that value is
>referenced using a pointer or whatever in computer theory - any link
>between pointers/references and mutability should be related to the
>implementation of the language - not the semantics.
Assignment, and therefore parameter passing, is entirely consistent
between mutable/immutable types. That is to say, assignment and
mutability are orthogonal and unrelated concepts in Python, just as you
suggest they should be. Perhaps some folks who only partially understand
Python's parameter passing semantics would be surprised by:
>>> def f(x):
... x = [4, 5, 6]
...
>>> a = [1, 2, 3]
>>> f(a)
>>> print a
[1, 2, 3]
Note that the assignment within f did not affect a in any way despite
the fact that a was an instance of a mutable type. It works exactly the
same way with a tuple.
With similar consistency you can call any method on an object passed as
a parameter. Where the distinction between mutable and immutable types
comes in is that there don't happen to be any methods of immutable
objects that actually modify the value of the object; mutable objects do
offer such methods. For example, tuples don't have "append" or "sort"
methods that modify the object's value in place, but lists do.
Now where some more of the assignment related confusion might come in is
with things like this:
>>> def f(x):
... x[0] = 10
...
>>> a = [1, 2, 3]
>>> f(a)
>>> print a
[10, 2, 3]
The body of f may look like it contains an assignment, but that's not
the typical name-binding that is referred to when discussing assignment
(or parameter passing) in Python. That is really just syntactic sugar
for calling a method of the parameter x, namely the __setitem__ method.
List objects are mutable and so, by definition, some methods like
__setitem__ will change their value. Tuples don't have __setitem__
methods so the above assignment in f won't work if you pass in a tuple.
Note that the same "assignment" outside of f wouldn't work for a tuple
either; this is a property of the tuple (i.e., it doesn't offer a
__setitem__ method), not of Python's parameter passing / assignment
semantics.
Jimmy
Jack Diederich
> On Sun, 2003-07-13 at 14:39, Stephen Horne wrote:
> > choice of programming language,
>
> the Python assignment statement. Python variables are not like C++
> variables, no surprise assignment is different too. If you used
> languages outside of C++ and its like (e.g., Pascal), you would find
> Python's behavior common.
>
C++ is the only language that has the same semantics as C++.
I was a long time C++ guy and my a-ha moment was when I realized that the
GoF's "Design Patterns" were not all universal -- some are C++ specific.
Trying to force Python semantics into a C++ world view will leave you feeling
empty every time. Drink the koolaid, python has a short learning curve.
If this is a problem - whatever you do - DONT TRY TO LEARN LISP.
If you want a more polished C++, use Java. Python is in a different class
of languages.
-jack
Martin v. Löwis
> >This just isn't true. The C++ assignment operator is not at all like
> >the Python assignment statement. Python variables are not like C++
> >variables, no surprise assignment is different too. If you used
> >languages outside of C++ and its like (e.g., Pascal), you would find
> >Python's behavior common.
[...]
> The ability to change part or all
> of a value in-place has nothing to do with whether that value is
> referenced using a pointer or whatever in computer theory - any link
> between pointers/references and mutability should be related to the
> implementation of the language - not the semantics.
So you think "assignment" is about "changing values"? This is the case
in C and C++, but not the case in Java (atleast for objects), and Python.
In Python, assignment changes variables, not values. This is something
fundamentally different. In Python (and many other languages),
variables are independent of their value (and vice versa). Variables
are *associated* with a value, instead of *being* that value. Then,
assignment changes that association - not the value itself.
> So much for dropping out of the discussion, but I hate it when people
> make false claims about my beliefs, making me out to be ignorant, when
> it is *not* *me* who is missing the point.
I'm uncertain what your point is, however, I do observe that computer
theory has a different view of what assignments are than what I think
your view is.
Regards,
Martin
Martin v. Löwis
> 1. Why are dictionarys called dictionaries (rather than references to
> dictionarys or whatever)? Similar for lists and class instances.
Because they are the values themselves, instead of being references to
values. These values (like all other values) are objects, though,
which means they have identity, state, and behaviour.
You might be thinking of values which only have state and
behaviour. Such values are not supported in Python - objects always
have an identity.
> 2. Why are the values of mutable objects always forced to be accessed
> via a pointer (or reference or whatever)?
Because all variables are references to objects. In Python, you cannot
have a variable that *is* a value - variables are always *bound* to
values, and accessing the variable yields the associated value. The
values exist independently from the variables they are bound to, and a
single value may be bound to different variables.
> 3. Why is there no way to reference an immutable object via a
> pointer, other than stuffing it into a mutable object designed for
> some purpose other than simple pointer behaviour?
But there is: If I do
a = 1
b = a
then b *refers* the same value that a refers to. That is, the values
associated with a and b have the same identity.
> This excuse is, as I already said, invalid.
Which excuse? You have only listed questions in this message, not
excuses.
Regards,
Martin
Bryan
> 3. Why is there no way to reference an immutable object via a
> pointer, other than stuffing it into a mutable object designed for
> some purpose other than simple pointer behaviour?
>
>>> a = (1, 2, 3)
>>> b = a
>>> id(a)
15471760
>>> id(b)
15471760
>>> print b[1]
2
>>>
i just referenced an immutable object via a "pointer" and i __did_not__
stuff it into a mutable object as you say.
a and b "point" to the same object.
bryan
Bengt Richter
>On 13 Jul 2003 10:37:01 -0400, aa...@pythoncraft.com (Aahz) wrote:
>
>>In article <iqm2hvs77lhe1m71v...@4ax.com>,
>>Stephen Horne <intent...@blank.co.uk> wrote:
>>>
>>>One of the few things I hate about Python is that mutable objects are
>>>implicitly shared using a reference system whereas immutable objects
>>>are not.
>>
>>Well, that's incorrect. *ALL* Python objects are implicitly shared with
>>bindings. The difference is whether updating the value referenced by a
>>target requires *re*binding the target or simply updating an object.
>
>Fine - nit-pick.
Well, it *is* a (very) significant nit ;-)
>
>All you have proven is that it is the distinction between types that
>get re-bound and those that don't (rather than the use of references)
>that is unnecessarily confusing and error prone.
I suspect you are reading some Python statements as if they were C or C++.
Python name assignment does not do anything to an object. It just creates
an alias name for an object. It may reuse an existing name, but that only
changes the meaning of that name as an alias, it does not affect the object
that the name was previously an alias for.
You can of course have other assignment targets than plain names. In fact,
a "plain name" is just the simplest expression for an assignment target.
But whatever the target expression evaluates to, what gets put "there" is
a reference to an object, not an object itself.
The confusion, ISTM, is in what to think of as "there." "There" is definitely
not a memory space for a Python object representation or "value".
The most accurate (UIAM) C concept would be a "there" of type PyObject* -- i.e.,
a pointer variable that points to the representation of some kind of Python object.
We don't have to know the implementation details to use the idea that the left hand
side of a Python assignment is an *expression* (even if a single name) that yields
a place to put a reference (effectively a pointer) to the object created by evaluating
the expression on the right hand side.
A C programmer will tend to think that a symbol on the left is a static expression
indicating a specific fixed memory space (static address or stack frame offset). But in Python
it is first of all a dynamic expression (though an unqualified target name can only have its "there"
be in the local or global namespace, and the choice between those is made statically
(at least w.r.t. a given level of compiling/exec-ing).
In C terms, the Python assignment target expression always evaluates to a place to put a pointer,
never a place to put object representation bits. Of course, "a place to put a pointer"
may be *inside* an object. And such inside places are identified by target expressions
such as x[2] or x.a or x[2]().b[3] etc. Even a bare name really does identify a place inside
an object -- i.e., inside the local or global dict associated with the context.
The "place" identified by x= after a global x declaration (or just in global scope) will be the
same place as globals()['x']= unless someone has subverted something. Either way, the binding
of the name to the object happens by evaluating to a "there" within the global dict object,
uniquely associated with the name ('x' here). Evaluated on the right hand side, that name will
produce the reference/pointer again for use in accessing the object or copying to another "there"
associated with perhaps some other name or a target within a composite object like a list or tuple,
or other namespace dict.
>
>The wart remains, even if my description was wrong. And even that is a
>dubious claim.
>
Every assignment is effectively stores a referential alias for an object,
whether associated with a name or not. This is by design, not a wart.
>A Python user is interested in how an object behaves - not how it is
>internally implemented in the interpreter. Immutable objects don't
when you say "immutable objects," are you really speaking of names
bound to immutable objects? I.e., the integer 123 is an immutable object
represented by the literal 123. You get an immutable object of the same value
from the expression int('123'). These are the int *objects* -- how does
"behave as references" apply to these actual "immutable objects"?
I.e., could you rewrite your sentence (that this is breaking in two)
to make it perhaps more understandable for me ?;-)
Names are not objects. Nor are left-hand-side assignment target expressions
in general, whether bare names or complicated.
ISTM what you are discussing is not about objects but about the way names work,
and ISTM you are thinking of names as if they were C++ variable references,
which they are not, and they couldn't be. For one thing, a C++ reference type
has to be initialized once to refer to a specific object. It can't be made to
refer to something else during the life of that scope. Pointers are a better
model, since you have to distinguish by expression syntax whether you mean
to assign a new pointer value or modify the thing pointed to. In python you
can only assign pointers, if you want to think in those terms. When you
modify a mutable, you are still assigning pointers into some part of the
mutable object representation. When you assign to a bare name, you are still assigning
a pointer into a place in some dictionary object (or workalike). If you
want to modify an object in the usual C sense, you can code it in C and provide a
python interface to your C-implemented object. When you pass arguments
to the various methods of your mutable, you will get pointers that you
can dereference and you can do what you want to your mutable's C data representation.
But when you pass the result back to the python world, it will have to be
as a standard object reference/pointer, and if you assign that, you will be
storing a pointer somewhere.
>behave as references - the internal use of references for immutable
>objects is basically a lazy copying optimisation and, apart from
>performace and a couple of other technicalities (e.g. the 'is'
>operator), has no relevance. Certainly it has no relevance to the
>point I was making.
>
The way it works is part of the semantics of the language, not just
an optimization issue.
Names aren't variables.
HTH ;-)
<disclaimer>I sometimes get things wrong. Corrections welcome.</disclaimer>
Regards,
Bengt Richter
Stephen Horne
Java has the same problem as Python in this respect. You can either
state that it implements a different assignment semantic for some data
types than it does for other data types, or you can claim that some
data types have implicit built in references while others don't.
The Java rationale is essentially that 'pointer' is considered a dirty
word, so it was considered necessary to disguise them in Java. End
result - you have to use things which are not pointers as if they
were. You have to abuse data types in an arbitrary, confusing and
error-prone way in order to create the effect of pointers/references.
In Python, Guido describes his rationale for the existence of both
mutable lists and immutable tuples here...
http://www.python.org/search/hypermail/python-1992/0292.html
According to this, the benefit from an immutable tuple is that there
is no immediate copy, yet the tuples stored in the object are
protected from accidental 'mutation'. It allows a lazy copy-on-write
optimisation combined with preventing unwanted mutation of those
tuples. ie Guido justifies the need for an immutable tuple in terms of
copying - the rationale is about assignment, even though the effect is
the same as in Java in which the rationale is about disguising
pointers.
Note that Guidos rationale assumes a tie between the concepts of
'reference to value' and 'in place modifiability of part or all of
value'. This is bogus - these are two different concepts. There is no
reason why mutable objects shouldn't be assigned using copy-on-write
semantics, and there is no reason why there shouldn't be a simple
reference or pointer data type which has the explicit purpose of
providing indirect reference to single values of any type whether
mutable or immutable.
The only 'trouble' is that it suggests a need to distinguish
assignment to a variable (replacing the previous value even if it was
a pointer) from assignment to a pointed-to location. In fact you
already do this if you fake pointer functionality using mutable
objects, the only difference is that a pointer dereference is specific
to the real purpose and does not require abuse of data types that are
designed for other jobs.
People may say that pointers are bad news, but that impression
resulted from languages like C and Pascal and it should be qualified
in two ways. Pointers in C and C++ are dangerous in a very important
way. Pointers don't always point to a valid object - they may point to
something that was deleted, for instance. This is worse in the C
family as pointers and arrays are almost treated as the same thing,
and array subscripting has no bounds checking. Furthermore, there is
the risk of memory leaks.
If pointers always point to a valid object (except, perhaps, a special
pointer-to-none value which throws an exception on dereferencing), if
pointers are kept distinct from other abstractions such as arrays, and
there is no possibility of accessing whatever memory happens to be
beyond the bounds of some particular type, and if garbage collection
is used then the only reliability objections that can be levelled at
pointers would relate to code complexity - and that complexity can
only be increased by faking pointers using mutable types designed for
other tasks. All of these are perfectly plausible for a scripting
language.
Donn Cave
...
| One way or the other, Python is currently choosing not to respect the
| computer science definition of those terms. It may have historic and
| backward-compatability reasons, but that does not change the facts.
| This deviation from computer science definitions, whatever the excuse,
| is arbitrary, confusing and error prone. That is my problem.
Well, there you have it - it does seem to be your problem. If we
were talking about a practical problem, we might be able to resolve
it with a few lines of code or an explanation of what really happens
when the code in question runs. But in this case, we all seem to
know perfectly well what happens, and how to write the code we need.
Python's system works. It's simple, practical and reasonably elegant.
If it doesn't work for you for some practical reason, I'd be surprised.
If you object to it on principle for some reason related to computer
science definitions, that really is your problem. I do suspect that
it would be also be due to some confusion on your part about either
how Python actually works, or what these computer science definitions
really say and how consistent and unambiguous they are, but from prior
experience with this type of thing I think it would be a mistake to
try to pursue it further here. It's computer science's job to describe
how Python works, not Python's job to match up with their terminology.
If you're one of those computer scientists, (or if you're not) you're
welcome to offer your interpretation.
Donn Cave, do...@drizzle.com
Fredrik Lundh
> Java has the same problem as Python in this respect. You can either
> state that it implements a different assignment semantic for some data
> types than it does for other data types, or you can claim that some
> data types have implicit built in references while others don't.
Or you can explain how it actually works, instead of making things up as
you post. In my experience, most people prefer that approach.
</F>
David Abrahams
> This is completely incorrect. On the contrary, builtin types are
> handled exactly uniformly in Python. The only difference here is that
> the builtin types that you've listed, along with some others, are
> immutable, so you cannot change them if you have a reference to them.
All you guys must be forgetting:
>>> a = [1]
>>> b = a
>>> a += b
>>> assert id(a) == id(b)
>>> a = 1
>>> b = a
>>> a += b
>>> assert id(a) == id(b)
Traceback...
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Aahz
Terry Reedy <tjr...@udel.edu> wrote:
>
>In Python, most builtin types are immutable. The two mutable builtin
>types are lists and dicts. Instances of user-defined classes are also
>mutable by default.
Actually, files are mutable, too. Then again, so are modules. And
functions. Gee, there are an awful lot of built-in mutable types. ;-)
Aahz
Ian Bicking <ia...@colorstudy.com> wrote:
>
>(Admittedly, some confusion may occur because these very different
>operations use the same syntax:
>
> x = 10
> x[0] = 10
> obj.x = 10
>
>The second and third are entirely different from the first.)
No, they aren't. They are precisely the same; they just have different
assignment targets.
Roy Smith
> I was a long time C++ guy and my a-ha moment was when I realized that the
> GoF's "Design Patterns" were not all universal -- some are C++ specific.
> Trying to force Python semantics into a C++ world view will leave you feeling
> empty every time.
Thank you for writing this, it makes me feel better!
I was having exactly this discussion just a couple of days ago with our
local hard-core C++ guy (who also poo-poo's Python, even though he's
never actually tried it). I made exactly the point Jack made -- that
the Design Pattern book is rather C++-centric, and some of it just
doesn't tranlate into Python very well.
For example, in the chapter on the Singleton pattern, a lot of time is
devoted to (what I consider) esoterica in the C++ access control system
(making copy constructors private, and all that stuff) to make sure
nobody can cheat and find a way to make a copy of the singleton. None
of that makes any sense in Python because (for better or worse), the
concept of private members just doesn't exist.
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>
>In Python, Guido describes his rationale for the existence of both
>mutable lists and immutable tuples here...
>
>http://www.python.org/search/hypermail/python-1992/0292.html
I'll note that Guido himself has also admitted that his ability to
explain his design decisions is far worse than his ability to select a
correct decision. These days, the rationale for tuples is that they're
a hyper-lightweight collection of heterogeneous values.
Aahz
David Abrahams <da...@boost-consulting.com> wrote:
>Erik Max Francis <m...@alcyone.com> writes:
>>
>> This is completely incorrect. On the contrary, builtin types are
>> handled exactly uniformly in Python. The only difference here is that
>> the builtin types that you've listed, along with some others, are
>> immutable, so you cannot change them if you have a reference to them.
>
>All you guys must be forgetting:
>
> >>> a = [1]
> >>> b = a
> >>> a += b
> >>> assert id(a) == id(b)
> >>> a = 1
> >>> b = a
> >>> a += b
> >>> assert id(a) == id(b)
> Traceback...
I'm not forgetting that. It's an unfortunate and tricky part of Python
semantics in some respects, but it's easily explained and understood if
you focus on the objects rather than the operation:
a = foo
b = a
a = a.__iadd__(b)
Where id(a)==id(b) depends solely on the behavior of __iadd__().
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>
>In computer science, a variable is a named binding to a value.
>Operations on that variable may rebind it to a different value, but
>values don't change behind your back. A pointer is, in effect, a value
>which indirectly refers to another (possibly anonymous) variable. A
>pointer is something you specifically request, and by doing so you
>allow that the 'variable' being indirectly referenced may be modified
>by something else that has no direct access to your pointer value (via
>your named variable or via copys or pointers-to your pointer held
>elsewhere).
>
>Python should respect that.
That's why I (and others) prefer to use "name" and "target" instead of
"variable". Would that assuage your ire?
Ian Bicking
this one...
On Sun, 2003-07-13 at 17:17, Martin v. Löwis wrote
> > 3. Why is there no way to reference an immutable object via a
> > pointer, other than stuffing it into a mutable object designed for
> > some purpose other than simple pointer behaviour?
>
> But there is: If I do
>
> a = 1
> b = a
>
> then b *refers* the same value that a refers to. That is, the values
> associated with a and b have the same identity.
I think, then, that Stephen maybe wants to do something like:
>>> a = ref(1)
>>> b = a
>>> b is a
True
>>> a + 2
3
>>> a = 5
>>> b
5
And so on. This is all quite doable in Python (implementation of ref
left to the reader), except for the "a = 5" part. Instead you'd have to
do something like "a.set(5)". Even though it is doable, of course, it's
by way of clever hacks.
The problem that Stephen is not appreciating is that this sort of
reference implies that "a" is a typed variable -- of type "pointer",
somehow implicitly declared when it was initially assigned a ref
object. (He's not alone, because just a little while ago someone
proposed a way to overload the meaning of "=", to achieve largely the
same functionality)
It might not be quite as ugly-seeming if we explicitly declared "a" as a
pointer. Besides the disinclination of people to add variable
declarations (though "global" already is one), it's more difficult
because there's now a new type that has to be usable everywhere, the
pointer type. All Python's code that expected, say, integers would have
to also be ready to dereference the argument and then use it. It could
probably be made seamless at the Python level fairly easily, but the C
would be annoying, no doubt.
One might think you could just use the references that already exists
(since variables are references, and everything is passed by
reference). But that's messed up, because that'd mean:
>>> b = 2
>>> pointer a
>>> a = 2
>>> a is b
True
>>> a = 5
>>> b
5
The ability to redefine the integers is obviously not a good feature
(though I remember someone saying you can accidentally do some stuff
like this in C extensions).
Anyway, that's my attempt at empathy with Stephen, if not agreement. If
he is wanting something other than what I've described, he should give
other examples, because code examples are better than words.
Ian
Stephen Horne
Appols for changing the subject while still staying in the same topic
- I just wanted to highlight this over my earlier boneheaded posts.
This is a post of heavily qualified appologies. In summary, I've been
thick-headed but I still believe that Python is doing the wrong thing.
I hope I've actually achieved a degree of sanity on this attempt ;-)
On 14 Jul 2003 00:17:19 +0200, mar...@v.loewis.de (Martin v. Löwis)
wrote:
>Stephen Horne <intent...@blank.co.uk> writes:
>
>> 1. Why are dictionarys called dictionaries (rather than references to
>> dictionarys or whatever)? Similar for lists and class instances.
>
>Because they are the values themselves, instead of being references to
>values. These values (like all other values) are objects, though,
>which means they have identity, state, and behaviour.
I've said my piece on meanings derived from computer science (and in
turn from mathematics, as it happens - variables as placeholders for
values predate electronic computers by quite some time.
People have tried to point out that copy-on-write is redundant for
immutable objects, but I've been too thick-headed. Sorry for that.
Even so, you can think of immutable objects as achieving 'expected'
assignment semantics simply because they are immutable. That does not
apply to mutable objects. So you can claim that Python behaviour is
consistent within itself, but only by claiming that it is consistently
different from this 'expected' behaviour.
Some people have already said that the word 'expected' is a fallacy -
that computer science doesn't have fixed definitions for these things.
Some have already named various languages which deviate in similar
ways in either some or all cases. But these languages are actually
deviating, either by convention or (in older cases) due to the
limitations of past machines. That's not the same thing as saying they
are doing 'the right thing' or that the computer science definitions
don't exist - all it means is that a lot of programmers who are
experienced in certain languages are familiar with this error-prone
system and used to compensating for its problems. But a modern
high-level language should do 'the right thing' - it should not be the
programmers job to make allowances for an outdated workaround designed
to give maximum performance by default on old machines irrespective of
reliability.
>> 2. Why are the values of mutable objects always forced to be accessed
>> via a pointer (or reference or whatever)?
>
>Because all variables are references to objects.
All variables are implemented as references, but that is not the same
as saying that all values are references. As Guidos own words reveal
(I already posted the link twice) the purpose of using immutables
tuples is performance optimisation to save copying while still
defending against against accidental mutation (ie it provides the
computer theory standard assignment semantics efficiently).
The same rationale should equally apply to all types, including the
mutable types.
In particular, it should apply to class instances. I shouldn't need to
create separate mutable and immutable variants of classes to get this
as the mutability of content should be the decision of the container
rather than the content. I shouldn't need to force copying of class
instances in the container for the exact same reason that Guido
highlights for tuples.
Guidos own example is actually quite bug prone in its own way. It
behaves differently depending on whether you pass in the point values
as tuples or lists. That is potentially a subtle, intermittent,
hard-to-trace error waiting to happen. You can say 'validation' all
you want. I did that back in the PEP238 days, and didn't get much
support. What's good for the goose is good for the gander.
>> This excuse is, as I already said, invalid.
>
>Which excuse? You have only listed questions in this message, not
>excuses.
Looking back, I was unfair to Tim Peters. The 'excuse' was percieved
(mainly because I'm frustrated trying to get my point across, and
because a circularity that arose (unless I was thick-headed about that
too) in another post. I said 'if a then b', someone said 'no - a is
wrong because c', I said 'ah, but if c then d' and the reply to that
was 'no - c is wrong because a' - or at least I think that's what
happened. Possibly I just need some sleep.
I thought Tim was also guilty of this, but looking back I was wrong.
Sorry, Tim.
However, revisiting his post I still disagree with it...
"""
It's deliberate and thoroughly consistent. Any assignment semantics
are
error-prone in some contexts, though (Python's included -- side
effects are
like that). The simplicity and absolute consistency of Python-style
assignment semantics are probably easiest for first-time programmers
to pick
up, and certainly dead easy for people coming from Lisp (etc) to pick
up --
people coming from C (etc) often struggle with inappropriate
expectations at
first.
"""
At first glance it looked like Tim was saying 'its all assignment
semantics rather than type semantics', hence my misjudging him. But
one way or the other, there is an inconsistency between the behaviour
of mutable objects and the computer science definitions of 'variable',
'value' and 'assignment' which does not arise for immutable objects
(note careful wording - see earlier apology). Having copy-on-write for
all objects (obviously the actual copying never happening for
immutables) would be both fully self-consistent and consistent with
the computer science definitions. The lack of this definitely does
cause both confusion and errors.
I also disagree with Tims statement that "Any assignment semantics are
error-prone in some contexts". OK, mistakes can be made with any
syntax for anything, but in the context of this discussion it is
perfectly possible to get the best of both worlds.
Make pointers/references explicit and make all assignments
'copy-on-write', and as I said earlier you get both mutability
protection by default for all types and the ability to reference any
type indirectly. Avoid the problems that make pointers in C, C++,
Pascal etc so unreliable and, while perceptions of pointers as being
inherently bug prone may persist they would be irrational - having
pointers explicitly for all types would at worst be as bug prone as
the existing situation where implicit references exist, and in
particular where mutable types get abused to fake pointer/reference
capabilities. It needs extra notation, but when mutables are used to
fake pointers there is extra notation anyway - misleading notation.
I'm not an anti-side-effect fanatic. I just don't want them to happen
by default, and I don't have to worry about this every time I use a
mutable type. And there really is no need for that to happen.
Stephen Horne
>Jack Diederich <ja...@performancedrivers.com> wrote:
>> I was a long time C++ guy and my a-ha moment was when I realized that the
>> GoF's "Design Patterns" were not all universal -- some are C++ specific.
>> Trying to force Python semantics into a C++ world view will leave you feeling
>> empty every time.
>
>Thank you for writing this, it makes me feel better!
>
>I was having exactly this discussion just a couple of days ago with our
>local hard-core C++ guy (who also poo-poo's Python, even though he's
>never actually tried it). I made exactly the point Jack made -- that
>the Design Pattern book is rather C++-centric, and some of it just
>doesn't tranlate into Python very well.
I thought that book uses Java (and UML)?
I agree with the point, though. A pattern is supposed to be a standard
way to implement an abstraction, but the best way to implement some
abstraction varies from language to language.
Stephen Horne
>In article <lq-cnWRG9rR...@comcast.com>,
>Terry Reedy <tjr...@udel.edu> wrote:
>>
>>In Python, most builtin types are immutable. The two mutable builtin
>>types are lists and dicts. Instances of user-defined classes are also
>>mutable by default.
>
>Actually, files are mutable, too. Then again, so are modules. And
>functions. Gee, there are an awful lot of built-in mutable types. ;-)
True. Damn. 'Boneheaded' is definitely my thing, today.
Stephen Horne
>In article <usmp9q...@boost-consulting.com>,
>>All you guys must be forgetting:
>>
>> >>> a = [1]
>> >>> b = a
>> >>> a += b
>> >>> assert id(a) == id(b)
>> >>> a = 1
>> >>> b = a
>> >>> a += b
>> >>> assert id(a) == id(b)
>> Traceback...
>
>I'm not forgetting that. It's an unfortunate and tricky part of Python
>semantics in some respects, but it's easily explained and understood if
>you focus on the objects rather than the operation:
I wish I'd thought of it, though - I could have avoided the
'boneheaded' confession a bit longer ;-)
Ian Bicking
> In article <mailman.105812672...@python.org>,
> Ian Bicking <ia...@colorstudy.com> wrote:
> >
> >(Admittedly, some confusion may occur because these very different
> >operations use the same syntax:
> >
> > x = 10
> > x[0] = 10
> > obj.x = 10
> >
> >The second and third are entirely different from the first.)
>
> No, they aren't. They are precisely the same; they just have different
> assignment targets.
Sure they are different. The first is a primitive operation binding the
variable x. The second gets x, and calls x.__setitem__(0, 10), and the
third is equivalent to setattr(obj, 'x', 10).
The first is primitive syntax. I suppose you could say that it could be
reduced to operations on locals() and globals(), but I feel like that's
a detail that is best not brought up ;) The local and global scope are
not as flexible as other objects The other two are really just syntactic
sugar.
Ian
Donn Cave
> In article <mailman.105812672...@python.org>,
|> Ian Bicking <ia...@colorstudy.com> wrote:
|>>
|>> (Admittedly, some confusion may occur because these very different
|>> operations use the same syntax:
|>>
|>> x = 10
|>> x[0] = 10
|>> obj.x = 10
|>>
|>> The second and third are entirely different from the first.)
|>
|> No, they aren't. They are precisely the same; they just have different
|> assignment targets.
|
| Sure they are different. The first is a primitive operation binding the
| variable x. The second gets x, and calls x.__setitem__(0, 10), and the
| third is equivalent to setattr(obj, 'x', 10).
I wonder if this is a good example of a mutability thing that really
is kind of missing to the detriment of Python.
If you want to support user-implemented sequences (as Python does)
but make Aahz's ``precisely the same'' assertion true in the sense
you're taking it - then I think the user-implemented indexing function
would have to return a reference to the index location.
class UserArray:
...
def __item__(self, index):
return &self.internal_array[index]
userArray[5] = 1
(python internal)
loc = UserArray.__item__(userArray, 5)
*loc = 1
if userArray[6] ==
(python internal)
loc = UserArray.__item__(userArray, 6)
return *loc
There are other places where instead we use some kludge with a
mutable container type, but in this case I don't think there's
a way to get around it. So whether we like the idea or not,
assignment to a user-implemented sequence doesn't have to involve
any assignment at all, because Python lacks the "pointer" type
(or "target" type, if you prefer) that would be required to
implement that.
Donn Cave, do...@drizzle.com
Stephen Horne
wrote:
>I think, then, that Stephen maybe wants to do something like:
>
>>>> a = ref(1)
>>>> b = a
>>>> b is a
>True
>>>> a + 2
>3
>>>> a = 5
>>>> b
>5
>
>And so on. This is all quite doable in Python (implementation of ref
>left to the reader), except for the "a = 5" part. Instead you'd have to
>do something like "a.set(5)". Even though it is doable, of course, it's
>by way of clever hacks.
Well, during the 'debate', that's the basic idea that involved -
except that I'd use explicit pointer-dereferencing syntax.
Imagine, for instance, changing all assignments and other 'copying' to
use a copy-on-write system. Then add a pointer type and a 'newcopyof'
operator. And nick the C-style prefix '*' for dereferencing and add
'&' as a 'make a pointer to this object' operator (absolutely NOT a
physical memory address).
>>> a = 5
>>> b = a
>>> b is a
False
>>> a = &5
>>> b = a
>>> *b is *a
True
>>> a = &5
>>> b = newcopyof a # could be "b = &(*a)" in principle
>>> *b is *a
False
>>> a = BadPtr
>>> b = *a
Traceback (most recent call last):
File "<stdin>", line 11, in ?
BadDeref: pointer does not reference a valid object
>>> a = *1
Traceback (most recent call last):
File "<stdin>", line 12, in ?
TypeError: object doesn't support dereference operator
No need for a 'delete' equivalent because of garbage collection. No
possibility of referencing a bad pointer as there is no way to create
a bad pointer - except, of course, for the explicit BadPtr (or
whatever) which throws an exception if dereferenced.
>The problem that Stephen is not appreciating is that this sort of
>reference implies that "a" is a typed variable -- of type "pointer",
>somehow implicitly declared when it was initially assigned a ref
>object. (He's not alone, because just a little while ago someone
>proposed a way to overload the meaning of "=", to achieve largely the
>same functionality)
No - I'm aware of the issue. When people use lists to fake pointers,
they already need explicit notation to tell Python what they are
doing. I'd just rather go back to having explicit pointers, though
implemented in a way which is appropriate to scripting languages and
which eliminates the major reliability issues of C-like pointer
mechanisms.
Using this approach, I don't need to declare variable types but I do
need to be explicit about when I'm using pointed-to-values and when
I'm using the pointers themselves.
Self-dereferencing pointers might also be nice in some ways, but with
serious problems. In C++, binding to the referenced object is done in
the declaration. That can't happen in Python, so a different method
would be needed - a special assignment operator, probably.
That then leads to the fundamental flaw. This couldn't be used to do
what, in C++, it does best - mimicking what Pascal calls 'var'
parameters.
If there were a 'procedure' abstraction, distinct from 'function', I
might be happy for all parameters to be implicitly dereferenced
pointers - I'd be happy being unable to change the pointer in this
case (in fact I'd like it enforced) - to get var parameter like
behaviour. There are nasty subtleties, though (what if someone creates
a pointer to the parameter, for instance - can it be used to change
the supposedly fixed pointer?). The C-like approach of just passing a
pointer as the expected parameter type has the advantage of simplicity
if nothing else.
>Anyway, that's my attempt at empathy with Stephen, if not agreement. If
>he is wanting something other than what I've described, he should give
>other examples, because code examples are better than words.
I hope I've explained, now. It wasn't in my mind when I started out,
but it would certainly be a way to solve what I see as serious
problems.
A realistic change for Python? - I doubt it. It could be technically
feasable, perhaps (a 'from future' thing adding copy-on-write and the
new operators etc) but interaction with past modules would create
problems. And with my very limited understanding of Python internals,
I'm sure I'm missing much bigger problems. Not to mention that at this
point, the change would probably be so fundamental as to actually
create a new language.
This does, however, give an alternative that *could* have been
plausible (in a parallel universe, perhaps), and which would not have
resulted in certain common problems that do occur now.
Stephen Horne
Technically, but what is the point? You can't do pointer-style things
with it. You can't change the object in any way without changing the
id, and you can't use the mechanism to, for instance, allow 'var'
parameters.
In short, you are showing evidence of the use of pointers internally
within Python, but that is not the same as providing pointer-like
semantics.
Adam Ruth
> On 13 Jul 2003 14:48:09 -0500, Ian Bicking <ia...@colorstudy.com>
> wrote:
>
> >On Sun, 2003-07-13 at 12:51, Stephen Horne wrote:
> 1. Why are the values of mutable objects always forced to be accessed
> via a pointer (or reference or whatever)?
Because *all* objects are accessed via pointer, sorta.
> Note that I am referring to a computer science, semantic
> pointer/reference/whatever - not to any pointers or references that
> may be used in the implementation of the binding of variables to
> values. The use, behind the scenes, of lazy copying as an optimisation
> is irrelevant to the computer science principles.
>
> 2. Why is there no way to reference an immutable object via a
> pointer, other than stuffing it into a mutable object designed for
> some purpose other than simple pointer behaviour?
They *are* referenced by pointer, sorta. It's just that what is being
pointed to cannot be changed. If you were using C and you had a
pointer to a variable in a region of memory that you couldn't change,
then you'd have, semantically, the equivalent.
In that case, how you do modify the value? You don't, you only modify
the *pointer*. If you want to change the pointer, you need to pass a
pointer to a pointer. This is what doesn't exist in Python, and I
think it's what you're having problems with. The same "problem"
exists in Java, but for slightly different reasons.
The Python method is completely consistent and all objects are treated
the same. There are no *pointers* even though binding sometimes
behaves like they. Binding doesn't always work like pointers, but it
always works like binding.
>
> The truth is that this system is arbitrary, and that therefore this
> excuse is invalid.
The definition of what is and isn't immutable isn't arbitrary in the
sense of "Determined by chance, whim, or impulse, and not by
necessity, reason, or principle". I'd say that the choices were made
on necessity, reason, and principle. Just not the necessity, reason,
or principle you would have chosen.
What happens in this case?
char *myval = "my little pony";
myval[1] = 'e';
It would appear that C and C++ also have immutable types. And in this
case the reason it's immutable bears no relation to the object or
type, but to it's location in memory, I'd say that that appears more
arbitrary. "Some strings are immutable and some aren't" is worse than
"all strings are immutable".
> >The problem you have is you are still thinking of variables as slots,
> >which is not correct. Variables in Python are bindings. Assignment
> >never copies anything, or creates anything except for changing the
> >variable to point to a different address location. *Every* Python
> >assignment (a=b) is like the C assignment (a=&b).
>
> The problem is that you are confusing implementation with semantics.
> The meanings of 'variable', 'value', 'assignment' etc are defined by
> computer science. Whether you arbitrarily change the meaning of
> 'assignment' or whether you arbitrarily change the meaning of
> 'variable', it amounts to the same thing.
I don't see the arbitrary change. Python has one type of variable: A
pointer that cannot itself be dereferenced. It's much more consistent
then having 3 types of variables (in C++).
>
> >
> >> Tell me one case where it is sensible for a function to behave such
> >> that whether the caller sees a change in a variable it passed as its
> >> argument should depend on the type.
A function may see a change in a *value* not a variable. That should
always depend on type and should depend on nothing else, a function
should always know what types it's working on. In my C example above,
it would depend on nothing that could be discernable in the function.
If someone wrote a mutable integer class, then it would be a distinct
type from an immutable integer, and the function could work with 3
types: immutable int, mutable int, and int. If the function doesn't
know what it's working with, then the function writer should probably
look for another career.
> You miss my point. Your argument would equally apply to the need for
> an integer-specific division operator, for instance. Its all about how
> one function reacts in response to varying input. If you pass a
> mutable value to a function which expects an immutable one, you get an
> error. If you pass an immutable value to a function which expects a
> mutable one, you get an error. There is no good reason for a function
> to react to the distinction as evidenced by your own observation that
> what actually happens (and what should happen) is that you have two
> distinct variants of the function.
Not true. If you pass a mutable to a function that expects an
immutable, you'll get no error. There's nothing that can be done to
an immutable that can't be done to a mutable. The opposite would give
an error, and should. You wouln't call a function in C++ with an
object that has no object::doSomething() function if the function
expected it. Whether an object is immutable or not, is no different
than knowing what methods it supports. You wouldn't program against
an object without knowing its methods, likewise you need to know if
it's immutable.
This dicussion brings to light the fact that immutable/mutable isn't
really special in any way. It just means that the object has no
methods to support mutability. Just like the classic Dog class
inheriting from Animal. Animal has no doBark() method, but Dog does.
A function user shouldn't be surprised that a method that expects a
Dog would choke on an Animal.
> >> Tell me one case where an object storing values should care about
> >> callers mutating values it holds *only* for certain types.
> >
> >Objects don't *store* values, they *refer* to values. You are still
> >thinking like you're in C (or C++). This is why you are having a
> >problem.
>
> No it is not. I'm thinking in terms of computer science, in which
> terms like 'variable', 'value' and 'assignment' are abstract concepts
> independent of the way in which they are implemented in a programming
> language.
The distinction is more to do with Dynamically version Static
languages. You're quoting the definitions for Static languages like
they apply universally to all computer science. They don't. In C you
can have a variable that's an int, float, pointer, struct, etc. In
Python you only get name (which can be read binding, or "limited"
pointer). It's when you start forcing the semantics of value types
onto variables that you get confused. Variable, value, and
assignement do mean different things in dynamic vs. static languages,
and that's because they're sematically different. While they both use
the = operator to perform similar functions, they're different, and if
you expect them to be identical, you're going to be frustrated.
> One way or the other, Python is currently choosing not to respect the
> computer science definition of those terms. It may have historic and
> backward-compatability reasons, but that does not change the facts.
> This deviation from computer science definitions, whatever the excuse,
> is arbitrary, confusing and error prone. That is my problem.
It's not respecting the static language definitions of those terms,
but it shouldn't even try. There are different meanings for those
words in mathematics, but you wouldn't try to express those
definitions in terms of C, it would be equally frustrating. The
static definitions are merely abstractions of the mathematic meanings,
and the dynamic meanings are also abstract, but in a different
direction.
Adam Ruth
Stephen Horne
>In article <sag3hv4vf884ed7ou...@4ax.com>,
>Stephen Horne <intent...@blank.co.uk> wrote:
>>
>>In computer science, a variable is a named binding to a value.
>>Operations on that variable may rebind it to a different value, but
>>values don't change behind your back. A pointer is, in effect, a value
>>which indirectly refers to another (possibly anonymous) variable. A
>>pointer is something you specifically request, and by doing so you
>>allow that the 'variable' being indirectly referenced may be modified
>>by something else that has no direct access to your pointer value (via
>>your named variable or via copys or pointers-to your pointer held
>>elsewhere).
>>
>>Python should respect that.
>
>That's why I (and others) prefer to use "name" and "target" instead of
>"variable". Would that assuage your ire?
Not really, for reasons defined elsewhere. The problem isn't
theoretical - it is practical, as shown by the fact that people do
trip over it time and time again. I'm just saying that the problem
arises out of peoples intuition of how variables should work - an
intuition that matches very well with theory.
If there is a clear concept of the right thing expressed in theory
which happens to match common intuitions, and if the wrong thing tends
to lead to subtle errors in functions that act differently depending
on whether the values passed in happen to be of mutable or immutable
types, all I can do is refer back to the arguments used against me
back in PEP238 when the majority *wanted* to make the massively
incompatible change on the grounds of theory, intuition and
reliability.
Stephen Horne
>I don't see the arbitrary change. Python has one type of variable: A
>pointer that cannot itself be dereferenced. It's much more consistent
>then having 3 types of variables (in C++).
I answered most of these arguments elsewhere (significantly with an
admission of stupidity) but your assertion here about C++ is wrong.
C++ has precisely one type of variable. That variable is a placeholder
for a value of a datatype which is specified in the declaration. The
datatype may be a pointer datatype, but so what? Pointer datatypes are
not treated any differently than other datatype except that they, like
all datatypes, they have their own set of functionality.
C++ references are tellingly also called self-dereferencing pointers.
They are not a distinct concept - they are syntactic sugar. I suspect
they mainly arise out of the modern desire to disguise pointers and
fantasize that they're not there, though they certainly work very well
in certain contexts.
>> One way or the other, Python is currently choosing not to respect the
>> computer science definition of those terms. It may have historic and
>> backward-compatability reasons, but that does not change the facts.
>> This deviation from computer science definitions, whatever the excuse,
>> is arbitrary, confusing and error prone. That is my problem.
>
>It's not respecting the static language definitions of those terms,
>but it shouldn't even try. There are different meanings for those
>words in mathematics, but you wouldn't try to express those
>definitions in terms of C, it would be equally frustrating. The
>static definitions are merely abstractions of the mathematic meanings,
>and the dynamic meanings are also abstract, but in a different
>direction.
Funny thing. When I use algebra, the variables I define don't end up
referring to different values, functions or whatever unless I
explicitly redefine them. When I write a definition on one piece of
paper, the things I wrote earlier on another sheet don't change.
Seems to me that the math equivalent of assignment (defining named
things) works very much like the 'static language definitions' as you
put it.
Martin v. Löwis
> >>>> a = ref(1)
> >>>> b = a
[...]
> >>>> a = 5
> >>>> b
> >5
[...]
> Well, during the 'debate', that's the basic idea that involved -
> except that I'd use explicit pointer-dereferencing syntax.
This is a very bad idea. You want to create a reference to the object
1? Then I assume that assigning to "a" would *change* the value of 1?
But 1 is an immutable object, it cannot change!
> Imagine, for instance, changing all assignments and other 'copying' to
> use a copy-on-write system.
That assumes that there is the notion of copying values in the first
place. Objects, in general, don't support copying - and assignment has
nothing to do with copying.
> >>> a = &5
> >>> b = newcopyof a # could be "b = &(*a)" in principle
> >>> *b is *a
> False
Again, objects don't support copying. For immutable objects (like
numbers), copying is also a pointless operation: If there where a
standard .copy method on objects, numbers would return themselves.
There is no way, in the language, to express that you want different
copies of the value 5. This is completely different from the notion of
values in C or C++, where each occurrence of the literal 5 creates a
new value whose state is 5.
Regards,
Martin
Martin v. Löwis
> I've said my piece on meanings derived from computer science (and in
> turn from mathematics, as it happens - variables as placeholders for
> values predate electronic computers by quite some time.
And so does mine: Variables are *not* containers of values, in
traditional mathematics. They are bound to values instead.
> Even so, you can think of immutable objects as achieving 'expected'
> assignment semantics simply because they are immutable. That does not
> apply to mutable objects. So you can claim that Python behaviour is
> consistent within itself, but only by claiming that it is consistently
> different from this 'expected' behaviour.
I am firmly convinced that your understanding of assignment is in
complete contradiction with traditional computer science, and the
notion of variables from a time that predates electronic computers.
So your expectation is wrong. Assignment does not mean that something
is copied, it means that something is *bound*.
> That's not the same thing as saying they are doing 'the right thing'
> or that the computer science definitions don't exist
You haven't given proof that these definitions do exist in "computer
science", and that the definitions match your understanding. To do so,
you would have to refer to some widely-used literature where a precise
definition of "variable" and "assignment" is given.
So far, you didn't show such proof - and anybody claiming that
literature fails to give proper definition, is inconsistent across
authors, or typically gives definitions that are different from your
understanding might be just as right as you might be.
> All variables are implemented as references, but that is not the same
> as saying that all values are references.
That's true. Values are not references, in Python. Values are values.
> As Guidos own words reveal (I already posted the link twice) the
> purpose of using immutables tuples is performance optimisation to
> save copying while still defending against against accidental
> mutation (ie it provides the computer theory standard assignment
> semantics efficiently).
>
> The same rationale should equally apply to all types, including the
> mutable types.
What is "the same" here: That mutable values should save copying when
being assigned? This is the case. That mutable values should be
protected against accidental mutation? That would be pointless - they
are mutable on purpose.
> I shouldn't need to force copying of class instances in the
> container for the exact same reason that Guido highlights for
> tuples.
It is a good thing that you need to invoke explicit code to perform
copying of objects. For objects, in general, you have the choice of
shallow or deep copies. Either may be meaningful, depending on the
context, and the language should not guess. Also, some objects (files,
windows) don't have a meaningful copy semantics in the first place.
Assignment has nothing to do with copying.
> Having copy-on-write for all objects (obviously the actual copying
> never happening for immutables) would be both fully self-consistent
> and consistent with the computer science definitions.
I can only repeat myself here: Assignment has nothing to do with
copying.
Regards,
Martin
Martin v. Löwis
> Funny thing. When I use algebra, the variables I define don't end up
> referring to different values, functions or whatever unless I
> explicitly redefine them. When I write a definition on one piece of
> paper, the things I wrote earlier on another sheet don't change.
That, typically, is because you don't have any mutating operations in
your formulae. All functions you use are side-effect-free. If you
translated the sheets of paper literally into Python, the formulae
would work the way you expect them to work.
Regards,
Martin
Bryan
> On Sun, 13 Jul 2003 22:42:21 GMT, "Bryan" <bel...@yahoo.com> wrote:
>
> >
> >> 3. Why is there no way to reference an immutable object via a
> >> pointer, other than stuffing it into a mutable object designed for
> >> some purpose other than simple pointer behaviour?
> >>
> >
> >>>> a = (1, 2, 3)
> >>>> b = a
> >>>> id(a)
> >15471760
> >>>> id(b)
> >15471760
> >>>> print b[1]
> >2
> >>>>
> >
> >
> >i just referenced an immutable object via a "pointer" and i __did_not__
> >stuff it into a mutable object as you say.
> >a and b "point" to the same object.
>
> Technically, but what is the point? You can't do pointer-style things
> with it. You can't change the object in any way without changing the
> id, and you can't use the mechanism to, for instance, allow 'var'
> parameters.
>
can you show what functionality is missing by not being able to do
"pointer-style things"? here are two variables:
a = (1, 2, 3)
b = [1, 2, 3]
one immutable, one mutable. please show an example of what "pointer-style
things" you would like to do.
i've read this entire thread so far, and from a practical point of view
(because that's all i really care about and what pays the bills), i really
don't understand what the problem is. personally, i don't care about the
strict definitions of computer science terms and what "variable" or
"pointer" means. i do care about how fast problems can be solved, how much
resources must be involved in solving that problem, and how much money can
be made/saved. for me, python is a winner in all this area. not having
pointers like c/c++ is such a blessing. i can't believe anyone would even
suggest that pointers and/or doing things with pointers be brought into
python. ok... one more time... let's bring this conversation down to
something practical. please show an example of pointer-style functionality
in any language that is missing/awkward/complicated in python. i'm not an
expert in python, i'm just very curious to understand what the true
practical issue is.
thanks,
bryan
Moshe Zadka
> The problem is that you are confusing implementation with semantics.
> The meanings of 'variable', 'value', 'assignment' etc are defined by
> computer science.
This is not just wrong, but as far as it is right, the semantics agree
with Python's :)
In computer science, the theoretical model one first learns is a turing
machine. Turing machines have no variables or values, merely tapes. While
I'm sure you can creatively translate the terms to turing machiens, this
exercise seems fairly pointless. Ram machines are similar. More abstract
models, like recursive functions, deal with integers only, so again you
would need hard to associate terms with them [you can implement a list
of integers with the p_1^n_1*....p_k^n_k, where p_1,...,p_k are distinct
primes. I'm not completely sure how this would translate to Python, though :)]
However, computer science did have a minor notational revolution: Lisp.
Lisp was originally invented as an abstract way to express computations
in a more natural, and yet well-defined, way. The fact that lisp can be
*implemented* was somewhat of a shock, but it turned to be a fairly useful
language.
So as far as computer science defines the idea of "variable", "value"
and "assignment", it can be said to give those terms the Lisp semantics,
which are fairly similar to Python semantics.
[Note that in Lisp conses, for example, are actually mutable. However,
many functions treat them as immutable objects. This is perfectly fine,
and was a concious design decision in Lisp. Similarily, Python wouldn't
*need* tuples: they can be implemented as lists. The immutability allows
the implementation to play tricks with memory usage, though.]
--
Moshe Zadka -- http://moshez.org/
Buffy: I don't like you hanging out with someone that... short.
Riley: Yeah, a lot of young people nowadays are experimenting with shortness.
Agile Programming Language -- http://www.python.org/
Erik Max Francis
> Imagine, for instance, changing all assignments and other 'copying' to
> use a copy-on-write system. Then add a pointer type and a 'newcopyof'
> operator. And nick the C-style prefix '*' for dereferencing and add
> '&' as a 'make a pointer to this object' operator (absolutely NOT a
> physical memory address).
The problem is that this misses the point that Python is not C++. You
shouldn't try grafting syntaxes and approaches that make Python look
more like C++, you should be learning to use Python on its own merits.
If what you want is a "pointer" -- something which encapsulates a
reference to some other object which can be changed -- then you can do
this very easily in Python through containment (either the
single-element list, or a dedicated, and thus probably more clear,
Container class like I gave at the outset of this thread). You're
saying you want something which misses the point of Python's handling of
variables: Variables are not slots where objects are placed, their
Post-It notes you can paste onto objects. Post-It notes can be added
and removed at will, and some names can be different in different
scopes.
Everything is handled the same way; it's just that some objects are
mutable in Python and some are not. You can say that immutability is
_equivalent_ to having those types of objects treated separately, but
that's not the same thing as them _actually_ being treated differently,
because as it's been made clear here they are not. You're drawing an
arbitrary line in the sand, but the line you're drawing is in the wrong
place (assignment doesn't work differently on mutable and immutable
objects -- with the exception of the extended += operators mentioned
earlier) and is not particularly arbitrary. Already examples of C code
that will choke based on their implicit immutability, mentioned nowhere
in the code by use of const, has been given; that makes C++'s mutability
issues far more arbitrary than Python's!
Or, to put it another way, if you want to program in C++, why not use
C++?
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ A life without festivity is a long road without an inn.
\__/ Democritus
Erik Max Francis
> All variables are implemented as references, but that is not the same
> as saying that all values are references.
Right. And this is an extremely important point in Python. Variables
are bindings of names to objects. Objects are entities in and of
themselves. The object is what is mutable or immutable, not the name.
Names are transitory.
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ The little I know, I owe to my ignorance.
\__/ Sacha Guitry
Michael Hudson
> On Sun, 13 Jul 2003 05:50:46 -0600, Dave Brueck
> <da...@pythonapocrypha.com> wrote:
>
> >But the method for returning more than one thing is not a simple progression
> >from this pattern. Instead you learn and shift to a *completely* different
> >mechanism. In Python, however, you *can* continue along the original route as
> >well:
> >
> >def foo(a, b, c):
> > return a+b, b+c
>
> You are not directly returning multiple values - you are returning a
> tuple. You can return containers in C++ too. The only difference is
> that Python makes working with tuples *much* easier - but its no more
> a 'simple progression' than in C++. You still need to know about
> tuples as well as returning values.
Actually, at one stage in Python's development it was almost
completely symmetric: you (always) passed in a tuple, and could
optionally return a tuple. The syntax made it look a bit like you
were passing in multiple arguments, but under the hood a tuple was
what you were passing, and there were ways in which this showed up.
IIRC,
f((a, b))
and
f(a, b)
were indistinguishable from inside f. This was all long before I came
across python, so I may be getting things mixed up...
Cheers,
M.
--
Roll on a game of competetive offence-taking.
-- Dan Sheppard, ucam.chat
Michael Hudson
> Is there someplace in particular to reference people who need to learn
> about what Python variables are (that they are bindings, etc)?
I like
http://starship.python.net/crew/mwh/hacks/objectthink.html
which has the advantage of doing things the pictorial (my) way, and
the verbal (Alex Martelli's) way.
Cheers,
M.
--
First time I've gotten a programming job that required a drug
test. I was worried they were going to say "you don't have
enough LSD in your system to do Unix programming". -- Paul Tomblin
-- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
I V
<intent...@blank.co.uk> wrote:
>The fact is that 'assignment' has a common meaning separate from the
>choice of programming language, and that the way Python handles name
Yes.
>binding means that meaning is not respected by Python for mutable
>objects.
No.
Assignment takes a name and binds it to a different value, whatever
programming language you use. Python does this for both immutable and
mutable types. The same is true of Lisp, and of C++ (although, in C++,
you can provide an implementation of operator= which does something
else, just as you can provide an implementation of operator+ which
doesn't add, but in both cases to do so is a mistake).
>Python, however, does not respect this meaning. It uses an arbitrary
>distinction based on whether or not the object has internals that can
>be usefully in-place modified, and uses that distinction to vary the
No, it doesn't.
x = (1, 2)
binds the name 'x' to a tuple entity with the value '(1, 2)'.
x = [1, 2]
binds the name 'x' to a list entity with the value '[1, 2]'.
There's no distinction here, arbitary or otherwise. There is an
exposed implementation detail as regards the identity of immutable and
mutable entities, but this doesn't effect the semantics of their
values.
--
"Mr I V Lenin, the Lenin of love"
Michael Chermside
(yet another) huge discussion of how assignment and parameter passing
works in Python.
In response, Ian Bicking wrote:
> Python is not novel in the way it deals with variables. Scheme and
> Smalltalk, for instance, act exactly the same, as do many other
> dynamically typed languages [...].
> This isn't a funny little feature, this is the way all strong,
> dynamically typed languages work.
[...]
> The problem you have is you are still thinking of variables as slots,
> which is not correct. Variables in Python are bindings. Assignment
> never copies anything, or creates anything except for changing the
> variable to point to a different address location. *Every* Python
> assignment (a=b) is like the C assignment (a=&b).
I have nothing whatsoever to add to this... it is just so clearly and
simply stated that I felt it was worth repeating. Of course, lots of
other people said the same thing in other ways, but I felt that Ian's
phrasing is particularly useful in explaining things to programmers
coming from a C-style background, even if it doesn't seem to reach
Stephen Horne.
-- Michael Chermside
Aahz
You've got a good point, but I think my point is just as valid. Which
viewpoint is more appropriate depends on the circumstance.
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>On 13 Jul 2003 21:43:41 -0400, aa...@pythoncraft.com (Aahz) wrote:
>>In article <sag3hv4vf884ed7ou...@4ax.com>,
>>Stephen Horne <intent...@blank.co.uk> wrote:
>>>
>>>In computer science, a variable is a named binding to a value.
>>>Operations on that variable may rebind it to a different value, but
>>>values don't change behind your back. A pointer is, in effect, a value
>>>which indirectly refers to another (possibly anonymous) variable. A
>>>pointer is something you specifically request, and by doing so you
>>>allow that the 'variable' being indirectly referenced may be modified
>>>by something else that has no direct access to your pointer value (via
>>>your named variable or via copys or pointers-to your pointer held
>>>elsewhere).
>>>
>>>Python should respect that.
>>
>>That's why I (and others) prefer to use "name" and "target" instead of
>>"variable". Would that assuage your ire?
>
>Not really, for reasons defined elsewhere. The problem isn't
>theoretical - it is practical, as shown by the fact that people do
>trip over it time and time again. I'm just saying that the problem
>arises out of peoples intuition of how variables should work - an
>intuition that matches very well with theory.
People trip over pointers, too, even when they're explicit. Heck,
people have problem with the simple
a = a + 1
no matter *what* programming language is used if they can't get past
their high school algebra and understand that "=" is a *command* in this
context. The question is not whether newcomers get tripped up, but the
extent to which the language can be easily defined in a consistent and
rigorous manner. Python passes that test quite handily, IMO.
Aahz
Stephen Horne <intent...@blank.co.uk> wrote:
>
>C++ has precisely one type of variable. That variable is a placeholder
>for a value of a datatype which is specified in the declaration. The
>datatype may be a pointer datatype, but so what? Pointer datatypes are
>not treated any differently than other datatype except that they, like
>all datatypes, they have their own set of functionality.
Really? I would argue that the differing syntaxes is an argument
against a single type of variable. What happens with this:
char a;
a->b;
Bengt Richter
[...]
Re-reading, I see that I should probably fill in some details (still subject to
disclaimer ;-)
>
>Python name assignment does not do anything to an object. It just creates
>an alias name for an object. It may reuse an existing name, but that only
>changes the meaning of that name as an alias, it does not affect the object
>that the name was previously an alias for.
>
>You can of course have other assignment targets than plain names. In fact,
>a "plain name" is just the simplest expression for an assignment target.
>But whatever the target expression evaluates to, what gets put "there" is
>a reference to an object, not an object itself.
>
>The confusion, ISTM, is in what to think of as "there." "There" is definitely
>not a memory space for a Python object representation or "value".
>
>The most accurate (UIAM) C concept would be a "there" of type PyObject* -- i.e.,
>a pointer variable that points to the representation of some kind of Python object.
>
I should have mentioned that there are various access mechanisms for storing the "pointer".
I.e., the left hand side doesn't evaluate to a raw address until you are in the machine language
of the implementation. When storage is actually happening, of course byte codes are being
interpreted by the virtual machine of the current Python version. The byte codes will
differ according to the translation of the target expression, and even for byte codes
that looks the same, the implementation may involve complex dynamic behavior, e.g., in
searches through an inheritance graph for an appropriate method that will accept the
rh object reference and do the ultimate implementation-pointer storage.
The byte codes with STORE in their names, and some typical source statements that generate them,
are (2.2.3 windows):
STORE_SLICE: x[a:b]=y # four byte codes numbered as STORE_SLICE+(1 if a is present)+(2 if b is present)
STORE_SUBSCR: x[a]=y
STORE_NAME: x=y #in global scope
STORE_ATTR: x.a=y
STORE_GLOBAL: def foo(): global x; x=y
STORE_FAST: def foo(): x=y
STORE_DEREF: def foo():
x=y # the STORE_DEREF
def bar():return x
return bar
Note that, e.g., STORE_SUBSCR looks the same for lists and dictionaries, or even something
undetermined that will give a run time error because the type of x doesn't support that operation.
Storage is often mediated by various methods, e.g., __setitem__ but finding the ultimate
method may be the result of a search through an inheritance graph,
and/or it could find the setter function of a named property, and that function in turn
may do much work that is not visible in the assignment statement source, or the immediate
level of corresponding byte codes.
>We don't have to know the implementation details to use the idea that the left hand
>side of a Python assignment is an *expression* (even if a single name) that yields
>a place to put a reference (effectively a pointer) to the object created by evaluating
>the expression on the right hand side.
Again, I should have been clearer. It *ultimately* yields a place, but potentially deep
in the bytecode and ultimately machine code of some chain of method invocations and implementations.
>
>A C programmer will tend to think that a symbol on the left is a static expression
>indicating a specific fixed memory space (static address or stack frame offset). But in Python
>it is first of all a dynamic expression (though an unqualified target name can only have its "there"
>be in the local or global namespace, and the choice between those is made statically
Hm. There is a third choice. When variables of the local namespace are destined to be captured
in a closure, the "there" is in the closure, stored by way of STORE_DEREF.
>(at least w.r.t. a given level of compiling/exec-ing).
>
>In C terms, the Python assignment target expression always evaluates to a place to put a pointer,
I elided a fair abount of implementation detail in saying that, but I think the effective
semantics are ok.
>never a place to put object representation bits. Of course, "a place to put a pointer"
>may be *inside* an object. And such inside places are identified by target expressions
>such as x[2] or x.a or x[2]().b[3] etc. Even a bare name really does identify a place inside
>an object -- i.e., inside the local or global dict associated with the context.
>
>The "place" identified by x= after a global x declaration (or just in global scope) will be the
>same place as globals()['x']= unless someone has subverted something. Either way, the binding
>of the name to the object happens by evaluating to a "there" within the global dict object,
>uniquely associated with the name ('x' here). Evaluated on the right hand side, that name will
>produce the reference/pointer again for use in accessing the object or copying to another "there"
>associated with perhaps some other name or a target within a composite object like a list or tuple,
>or other namespace dict.
>
>>
>>The wart remains, even if my description was wrong. And even that is a
>>dubious claim.
>>
>Every assignment is effectively stores a referential alias for an object,
>whether associated with a name or not. This is by design, not a wart.
>
The details of *how* an assignment "effectively stores" is a longer story though ;-)
>>A Python user is interested in how an object behaves - not how it is
>>internally implemented in the interpreter. Immutable objects don't
>when you say "immutable objects," are you really speaking of names
>bound to immutable objects? I.e., the integer 123 is an immutable object
>represented by the literal 123. You get an immutable object of the same value
>from the expression int('123'). These are the int *objects* -- how does
>"behave as references" apply to these actual "immutable objects"?
>I.e., could you rewrite your sentence (that this is breaking in two)
>to make it perhaps more understandable for me ?;-)
>
>Names are not objects. Nor are left-hand-side assignment target expressions
>in general, whether bare names or complicated.
>
>ISTM what you are discussing is not about objects but about the way names work,
>and ISTM you are thinking of names as if they were C++ variable references,
>which they are not, and they couldn't be. For one thing, a C++ reference type
>has to be initialized once to refer to a specific object. It can't be made to
>refer to something else during the life of that scope. Pointers are a better
>model, since you have to distinguish by expression syntax whether you mean
>to assign a new pointer value or modify the thing pointed to. In python you
>can only assign pointers, if you want to think in those terms. When you
>modify a mutable, you are still assigning pointers into some part of the
>mutable object representation. When you assign to a bare name, you are still assigning
>a pointer into a place in some dictionary object (or workalike). If you
>want to modify an object in the usual C sense, you can code it in C and provide a
>python interface to your C-implemented object. When you pass arguments
>to the various methods of your mutable, you will get pointers that you
>can dereference and you can do what you want to your mutable's C data representation.
>But when you pass the result back to the python world, it will have to be
>as a standard object reference/pointer, and if you assign that, you will be
>storing a pointer somewhere.
>
>>behave as references - the internal use of references for immutable
>>objects is basically a lazy copying optimisation and, apart from
>>performace and a couple of other technicalities (e.g. the 'is'
>>operator), has no relevance. Certainly it has no relevance to the
>>point I was making.
>>
>The way it works is part of the semantics of the language, not just
>an optimization issue.
>
>Names aren't variables.
>
>HTH ;-)
>
><disclaimer>I sometimes get things wrong. Corrections welcome.</disclaimer>
>
Regards,
Bengt Richter
Andrew Dalke
> In computer science, the theoretical model one first learns is a turing
> machine. Turing machines have no variables or values, merely tapes.
Actually, we started off with state machine, then worked our way though
DFA/NFA, PDA, etc., ending up with a TM. I remember being intrigued
that the way we learned these was in opposite order to they way they
were first researched.
Can anyone suggest why? My thought was that mathematicians
like generalizing, so the most general models were pondered first.
Only as people started to experiment with variations, and with actual
computer hardware to back them up, did people start thinking about
more limited cases.
Andrew
da...@dalkescientific.com
Hannu Kankaanpää
> >def foo(a, b, c):
> > return a+b, b+c
>
> You are not directly returning multiple values - you are returning a
> tuple. You can return containers in C++ too. The only difference is
> that Python makes working with tuples *much* easier - but its no more
> a 'simple progression' than in C++. You still need to know about
> tuples as well as returning values.
You don't need to know about tuples so that you can understand
that multiple values are returned, and that's the difference.
I didn't know of tuples when I saw some of my first Python code,
yet the idiom for returning multiple values was trivially clear.
It became even better though when I learned it was implemented
through tuples ;).
In contrast, in C++ one needs to explicitely state make_pair
or make_tuple, requiring the user to know of the underlying
types.
Had I been required to type
def foo(a, b, c):
return tuple(a+b, b+c)
x = foo(a, b, c)
a = x[0]
b = x[1]
Then I could've agreed with you. But multi-assignment from
tuples and implicit creation of tuples makes the returning
of multiple values look like a language feature, not a way
to abuse tuples. This ease of use is the reason it can work
as a replacement for C++'s "send-by-reference".
Bengt Richter
>On Sun, 2003-07-13 at 20:32, Aahz wrote:
>> In article <mailman.105812672...@python.org>,
>|> Ian Bicking <ia...@colorstudy.com> wrote:
>|>>
>|>> (Admittedly, some confusion may occur because these very different
>|>> operations use the same syntax:
>|>>
>|>> x = 10
>|>> x[0] = 10
>|>> obj.x = 10
>|>>
>|>> The second and third are entirely different from the first.)
>|>
>|> No, they aren't. They are precisely the same; they just have different
>|> assignment targets.
>|
>| Sure they are different. The first is a primitive operation binding the
>| variable x. The second gets x, and calls x.__setitem__(0, 10), and the
>| third is equivalent to setattr(obj, 'x', 10).
Which might turn out to be equivalent to object.__setattr__(obj, 'x', 10)
or obj.__setattribute__('x', 10) or obj.__class__.__dict__['x'].fset(obj, 10) or ...
It depends on what you mean by 'targets'. I argued that the three left hand
expressions all *ultimately* result in some machine level pointer variable being set
in some object's internal representation to point to the representation of the object
resulting from evaluating the right hand side, and in that sense all the assignments
have the same ultimate semantics. Some pointer in some composite object is set to point
to another object.
Depending on the left hand object actually involved, we use different expressions to
retrieve the pointer (and often you have a choice, e.g., x.y or x.__dict__['y'] or getattr(x,'y')
or vars(x).get('y') etc., some of which might not be valid. Even plain x might sometimes be
retrieved by sys._getframe().f_locals.get('x').
>
>I wonder if this is a good example of a mutability thing that really
>is kind of missing to the detriment of Python.
>
>If you want to support user-implemented sequences (as Python does)
>but make Aahz's ``precisely the same'' assertion true in the sense
>you're taking it - then I think the user-implemented indexing function
>would have to return a reference to the index location.
>
I guess you could define something like your *loc below, but you'd have to be
willing to spell it loc.star when you used it.
class Loc(object): #XXX# untested !!
def __init__(self, theArray, theItem=0): self._theArray=theArray; self._theItem=theItem
def _get_it(self): return self._theArray[self._theItem]
def _set_it(self,v): self._theArray[self._theItem] = v
star = property(_get_it, _set_it)
> class UserArray:
> ...
> def __item__(self, index):
return Loc(self, index)
> return &self.internal_array[index]
>
> userArray[5] = 1
> (python internal)
> loc = UserArray.__item__(userArray, 5)
loc.star = 1
> *loc = 1
>
> if userArray[6] ==
> (python internal)
> loc = UserArray.__item__(userArray, 6)
return loc.star
> return *loc
>
>There are other places where instead we use some kludge with a
>mutable container type, but in this case I don't think there's
>a way to get around it. So whether we like the idea or not,
>assignment to a user-implemented sequence doesn't have to involve
>any assignment at all, because Python lacks the "pointer" type
>(or "target" type, if you prefer) that would be required to
>implement that.
Or you could say the implementation is internal and you have
to use the right spellings to effect the semantics indirectly.
I.e., op[i]=newitem spells *p = newitem internally (names obviously not directly related):
int
PyList_SetItem(register PyObject *op, register int i,
register PyObject *newitem)
{
...
p = ((PyListObject *)op) -> ob_item + i;
olditem = *p;
*p = newitem;
Py_XDECREF(olditem);
return 0;
}
Regards,
Bengt Richter
Adam Ruth
> C++ has precisely one type of variable. That variable is a placeholder
> for a value of a datatype which is specified in the declaration. The
> datatype may be a pointer datatype, but so what? Pointer datatypes are
> not treated any differently than other datatype except that they, like
> all datatypes, they have their own set of functionality.
Granted. Pointers are no different than other data types, but they
are typically used for operations that are semantically very different
than other datatypes are used for. In that sense, they are, at a high
level, different data types.
It's like how c programming is taught has having pass by reference and
pass by value, when there is only pass by value at the implementation
level. Pass by reference is a concept added on top of the language.
> C++ references are tellingly also called self-dereferencing pointers.
> They are not a distinct concept - they are syntactic sugar. I suspect
> they mainly arise out of the modern desire to disguise pointers and
> fantasize that they're not there, though they certainly work very well
> in certain contexts.
Syntactic sugar or no, they still behave differently than other
datatypes and are therefore not consistent... IMHO.
> Funny thing. When I use algebra, the variables I define don't end up
> referring to different values, functions or whatever unless I
> explicitly redefine them. When I write a definition on one piece of
> paper, the things I wrote earlier on another sheet don't change.
>
> Seems to me that the math equivalent of assignment (defining named
> things) works very much like the 'static language definitions' as you
> put it.
The devil is in the details. Math assignment is static assignment is
dynamic assignment. They really are all the same thing at a high
level, but it's the implementation and the subtleties that make them
vary.
Adam Ruth
Adam Ruth
> I've said my piece on meanings derived from computer science (and in
> turn from mathematics, as it happens - variables as placeholders for
> values predate electronic computers by quite some time.
The mathematics I'm used to has a very different concept of assignment
than does a static language like C. Here's why.
In math there are global unchanging objects called numbers, 1, 2, 3,
etc. If I have 2 variables, x and y and they both equal 500, then
they both are names bound to the value 500. They are not each copies
of the value 500, math has no such concept, there is only one 500.
C, and most static languages, have the concept of memory location,
which doesn't exist in math. In C a variable is bound to a memory
location which contains a value, not like math where the variable
itself contains the value. Therefore I can have two variables, x and
y, that point to their own distinct copy of 500 and when I assign a
value, it copies the 500 into the memory location bound to x.
Now, what if I have x and y bound the the same memory location? If I
change X I also change Y. That's not how mathematics works. If X and
Y are to have to same value, they do so because of some relation to
their meaning, and they are both bound to 500, not simply because I
changed the value of X. This is a concept that simply doesn't exist
in mathematics.
Python is more like the math form. There is only one instance of the
number 500 and a name (variable) is bound to it or it isn't. To
change the value of 500, then I must rebind the name, exactly in
mathematics. However, Python differs from math in that math doesn't
have mutable values (well it may when you get into real complex
calculus, I don't know, never been there).
Adam Ruth
Donn Cave
wrote:...
> Which might turn out to be equivalent to object.__setattr__(obj, 'x', 10)
> or obj.__setattribute__('x', 10) or obj.__class__.__dict__['x'].fset(obj, 10)
> or ...
>
> It depends on what you mean by 'targets'. I argued that the three left hand
> expressions all *ultimately* result in some machine level pointer variable
> being set
> in some object's internal representation to point to the representation of
> the object
> resulting from evaluating the right hand side, and in that sense all the
> assignments
> have the same ultimate semantics. Some pointer in some composite object is
> set to point
> to another object.
>
> Depending on the left hand object actually involved, we use different
> expressions to
> retrieve the pointer (and often you have a choice, e.g., x.y or
> x.__dict__['y'] or getattr(x,'y')
> or vars(x).get('y') etc., some of which might not be valid. Even plain x
> might sometimes be
> retrieved by sys._getframe().f_locals.get('x').
My point really was that we can't express "pointer" or "target" in
Python, so Python can't "retrieve the pointer" from user defined
__setitem__ - it just has to hand it the values and assume that
an assignment will actually happen somewhere.
Not that it's worth making the necessary changes to Python, but it
does seem to me that it would have been more elegant and rigorous
to have indexing and assignment work that way, and it seems to me
that when we can't do it because we can't express "target", that
is a structural weakness in the language.
The Loc class you proposed as a solution doesn't really do that
at all - it's just a round-about way to call __setitem__, for all
I can tell.
> Or you could say the implementation is internal and you have
> to use the right spellings to effect the semantics indirectly.
> I.e., op[i]=newitem spells *p = newitem internally (names obviously not
> directly related):
>
> int
> PyList_SetItem(register PyObject *op, register int i,
> register PyObject *newitem)
> {
> ...
> p = ((PyListObject *)op) -> ob_item + i;
> olditem = *p;
> *p = newitem;
> Py_XDECREF(olditem);
> return 0;
> }
I'm missing something here - this evidently does not apply to user
defined __setitem__?
Donn Cave, do...@u.washington.edu
Moshe Zadka
> In computer science, the theoretical model one first learns is a turing
> machine. Turing machines have no variables or values, merely tapes.
[Andrew Dalke]
> Actually, we started off with state machine, then worked our way though
> DFA/NFA, PDA, etc., ending up with a TM.
Obviously, I meant "first turing-equivalent". Of course, whether this
is true also depends: in CS "Computability" this is true, but
in Math, we covered this in "Logic 2", and there we did, IIRC,
recursive functions and counting machines first. Counting machines
are mostly useful as something on the way to prove recursive functions
are turing complete.
> I remember being intrigued
> that the way we learned these was in opposite order to they way they
> were first researched.
>
> Can anyone suggest why? My thought was that mathematicians
> like generalizing, so the most general models were pondered first.
> Only as people started to experiment with variations, and with actual
> computer hardware to back them up, did people start thinking about
> more limited cases.
Well, I think that it's natural to study DFAs first, as a turing
machine is a DFA with a tape, so it's useful to know *why* the
tape is needed. Also, DFAs are a good explanation of "what are
computers", as a computer is necessarily finite.
Stephen Horne
>Stephen Horne <intent...@blank.co.uk> wrote in message news:<1br3hv4434gn7p2pd...@4ax.com>...
>> I've said my piece on meanings derived from computer science (and in
>> turn from mathematics, as it happens - variables as placeholders for
>> values predate electronic computers by quite some time.
>
>The mathematics I'm used to has a very different concept of assignment
>than does a static language like C. Here's why.
>
>In math there are global unchanging objects called numbers, 1, 2, 3,
>etc. If I have 2 variables, x and y and they both equal 500, then
>they both are names bound to the value 500. They are not each copies
>of the value 500, math has no such concept, there is only one 500.
Yes.
Mathematics does have assignment - it had assignment long before
electronic computers existed because algorithms were a concept studied
in mathematics long before computers existed.
If in mathematics, if I have one variable a defined as representing
the set {1, 2, 3} and a variable y also defined as representing the
set {1, 2, 3} and I decide to change y to {1, 2, 4} then that
redefinition process does not change the value {1, 2, 3} to {1, 2, 4}.
It only changes what the variable y is currently taken to represent.
Values cannot be mutable in mathematics.
In Python, *objects* can be mutable. Values cannot be mutable because
they are values - something defined in mathematics. 1 cannot become 2.
But a variable can be reassigned to represent something else. In-place
modification of part of an object is a practical way of replacing one
value with another.
Thus, if I write...
>>> a=[1,2,3]
>>> a[2]=4
>>> a
[1, 2, 4]
Nothing is wrong. The in-place modification of part of the object that
a is bound to is basically a convenient shorthand for reassigning the
whole value of a. The value now bound to a is [1, 2, 4] rather than
[1, 2, 3]. No value has mutated. Nothing is wrong in mathematical
terms. The fact that we are using something called an object (or a
pointer to an object or whatever) to implement the binding to a value
is unimportant.
>>> a=[1,2,3]
>>> b=a
>>> b[2]=4
>>> b
[1, 2, 4]
>>> a
[1, 2, 4]
Because the variable b has been bound to an object (*NOT* a value),
the mutation has a side-effect. Changing the value bound to the
variable b also changes the value bound to a. This does *NOT* happen
in mathematics, because there is no such thing as an object.
Reassigning one variable in mathematics never implicitly redefines
another.
In short, this no longer implements a binding of values to variables
because of the side-effect of changing the value bound to a when you
change the value bound to b. The implementation detail of how the
binding from variables to values is achieved (by binding to objects)
has changed the results of running the algorithm.
In a true functional language, variables are not mutable. It doesn't
matter in the slightest if binding of variables is handled by copying
pointers to objects instead of copying objects - it is irrelevant
because copying pointers to objects achieves exactly the same thing as
copy the objects themselves - it implements a binding of value to
variable.
The object bound to a variable in Python is a low-level implementation
concept. From Guidos own words, we can see that he (at least at one
point) recognised that binding to objects in a context where those
objects can be mutable is confusing and error-prone.
The value bound to a function in mathematics is a high-level concept.
I thought Python was *supposed* to be an very high-level language?
Even C++ is higher level that this. The C++ standard libraries I've
used will optimise assignment of strings by merely copying pointers.
The sequence of events you get is something like this...
std::string x ("Hello");
// x bound to object containing value "Hello"
std::string y (x);
// y bound to same object as x, but with an indication that
// sharing is happening.
y [4] = '!';
// A write to y, with the purpose of changing the value that y
// is bound to but not changing the value that x is changed to.
// Thus a copy of the object containing "Hello" is made and
// bound to y before applying the change. This is called
// copy-on-write.
In short, C++ can and often does use the pointer-copying optimisation
in the standard library. But it does so in such a way that the low
level binding of variables to objects implements the high level
binding of variables to values correctly.
It is clear now that many people are highly committed in the binding
of variables to objects done in a way that does not implement binding
of variables to values - though there seems to be fuzzy thinking used
against me just as much as my own fuzzy thinking earlier. For
instance...
On 14 Jul 2003 07:39:57 +0200, mar...@v.loewis.de (Martin v. Löwis)
wrote:
: And so does mine: Variables are *not* containers of values, in
: traditional mathematics. They are bound to values instead.
Python does not bind values to variables. It binds objects to
variables. Objects are how values are implemented, but they are not
the same thing.
In computer science and mathematics, the whole point of variables is
to bind a variable to a *value*. Terms such as 'store location' and
'object' are about *implementing* that binding from variable to value.
In mathematics, it doesn't matter whether you talk of variables being
placeholders for values or whether you talk of them being bound to
values. The two are equivalent, because mathematics has no concept of
objects as an implementation detail. Both terms are in common use.
In support of this, I quote someone arguing against me...
On 14 Jul 2003 11:44:02 -0700, ow...@hotmail.com (Adam Ruth) wrote:
"""
C, and most static languages, have the concept of memory location,
which doesn't exist in math. In C a variable is bound to a memory
location which contains a value, not like math where the variable
itself contains the value. Therefore I can have two variables, x and
y, that point to their own distinct copy of 500 and when I assign a
value, it copies the 500 into the memory location bound to x.
"""
"not like math where the variable itself CONTAINS the value"
In addition, note that C++ does not copy the value. It copies a
representation of the value. Thus when you say two variables contain
"their own distinct copy of 500" what you are really saying is that
two variables contain "their own distinct copy of a representation of
the value 500". That is why you can use the operator '==' to test if
two variables contain the same value - or rather in-store
representations that represent the same value.
"""
In math there are global unchanging objects called numbers, 1, 2, 3,
etc. If I have 2 variables, x and y and they both equal 500, then
they both are names bound to the value 500. They are not each copies
"""
No - they are not objects, or store locations. They are global
unchanging values. And yes, variables *are* bound to values - *not*
objects.
"""
Python is more like the math form. There is only one instance of the
number 500 and a name (variable) is bound to it or it isn't. To
change the value of 500, then I must rebind the name, exactly in
mathematics. However, Python differs from math in that math doesn't
have mutable values (well it may when you get into real complex
calculus, I don't know, never been there).
"""
Mutable values don't exist at all. Mutable objects do. When you mutate
a part of an object, that object now represents a new value.
Its a shorthand in Python for something that is perfectly valid in
mathematics - but not if the binding of other variables to values is
affected.
BTW - "real complex" is a contradiction in terms ;-)
Anyway, if you so love your variables binding to objects done in a way
that does not correctly implement variables binding to values for
mutable objects, fine. It simply means that Pythons users have to fuss
about the implementation of Python instead of getting on with their
applications logic, and that it will continue to cause confusion and
errors on an ongoing basis.
Stephen Horne
>>>That's why I (and others) prefer to use "name" and "target" instead of
>>>"variable". Would that assuage your ire?
>>
>>Not really, for reasons defined elsewhere. The problem isn't
>>theoretical - it is practical, as shown by the fact that people do
>>trip over it time and time again. I'm just saying that the problem
>>arises out of peoples intuition of how variables should work - an
>>intuition that matches very well with theory.
>
>People trip over pointers, too, even when they're explicit. Heck,
>people have problem with the simple
>
> a = a + 1
>
>no matter *what* programming language is used if they can't get past
>their high school algebra and understand that "=" is a *command* in this
>context. The question is not whether newcomers get tripped up, but the
>extent to which the language can be easily defined in a consistent and
>rigorous manner. Python passes that test quite handily, IMO.
Ah - but this is also Pythons fault. It has arbitrarily redefined the
meaning of a mathematics operator - presumably simply because C, C++,
Java, Basic and others have done this in the past.
C's rationale for this still amuses me. It was because there are more
assignments in a typical program than tests for equality. It was an
argument about numbers of keypresses!
Frankly, this is another thing that should not be done in a 'very high
level language'.
In BASIC, the operator is a shorthand for what was originally written
with a 'LET' command. Annoying, but at least the original form made
the point.
Personally, though, I'd go with Pascal on this one.
Similarly, if Pythons binding of variables to objects was used to
correctly implement a binding of variables to values - as happens in
mathematics - there'd be no confusion.
In another thread, hokiegal99 made a mistake where she expected the
string.strip method to literally change the name of a file on disk.
But was she being irrational? I don't think so. She had no doubt been
told, in effect, that Python variables bind to objects rather than
values. The object she was changing was 'the name of a file'. If
Python variables bind to objects rather than values, then that os.walk
method should be providing literally a list of the names of the files
- objects which literally refer back to the on disk file names. It
doesn't. It provides a list of representations of the names at a point
in time.
Of course in this case, binding to the object itself (or something
that implements a binding to that object) would be complex and
inefficient. I guess Python is just doing what is convenient for the
implementation.
Stephen Horne
wrote:
>Stephen Horne <intent...@blank.co.uk> writes:
Funny. Algorithms in mathematics have been around much longer than
Python or even electronic computers.
Stephen Horne
>In article <7dc4hvslh7a8fk9p6...@4ax.com>,
>Stephen Horne <intent...@blank.co.uk> wrote:
>>
>>C++ has precisely one type of variable. That variable is a placeholder
>>for a value of a datatype which is specified in the declaration. The
>>datatype may be a pointer datatype, but so what? Pointer datatypes are
>>not treated any differently than other datatype except that they, like
>>all datatypes, they have their own set of functionality.
>
>Really? I would argue that the differing syntaxes is an argument
>against a single type of variable. What happens with this:
>
>char a;
>a->b;
To answer this, I simply have to restore the quote you deleted...
: On 13 Jul 2003 21:03:59 -0700, ow...@hotmail.com (Adam Ruth) wrote:
:
: >I don't see the arbitrary change. Python has one type of variable: A
: >pointer that cannot itself be dereferenced. It's much more consistent
: >then having 3 types of variables (in C++).
This is not referring to data types - there are far more than three
data types in C++. It is referring to whether something is a pointer
or a reference or not. The mistake in Adams post simply being that
these are simply datatypes.
You will note that even in my words, I tried to keep the distinction
clear by using the word 'datatypes' when I was referring to datatypes.
In fact, lets look back at my first two scentences in your quote of my
reply.
>>C++ has precisely one type of variable. That variable is a placeholder
>>for a value of a datatype which is specified in the declaration.
One *type* of variable, which is associated with its own *datatype*.
I find it hard to see how you could confuse this, but given my own
mistakes - well, we're all human I suppose.
Tom Plunket
> All you have proven is that it is the distinction between types
> that get re-bound and those that don't (rather than the use of
> references) that is unnecessarily confusing and error prone.
As the firestarter (I apologize for the flames that have
erupted), I must admit that even I know that this distinction
does not exist in Python. There aren't some types that get
rebound and others that don't.
def fn(arg):
local = SOME_GLOBAL
local = arg
local gets rebound to arg regardless of what the types involved
are, period.
> The wart remains, even if my description was wrong. And even that
> is a dubious claim.
I still feel that there's a wart of some sort, but it's not due
to any inconsistency in the language, despite what I may have
said before (I've been thinking about it a lot over the past few
days).
> A Python user is interested in how an object behaves - not how it
> is internally implemented in the interpreter. Immutable objects
> don't behave as references - the internal use of references for
> immutable objects is basically a lazy copying optimisation and,
> apart from performace and a couple of other technicalities (e.g.
> the 'is' operator), has no relevance.
Are they really technicalities, though? I mean, a mutable object
can be changed. It's the same object, just changed. Immutable
objects, however, can't be changed. You can get something that
appears like changing it, but it's actually building a new thing
and binding the label to that new thing.
What I specifically want is a way to have a "second-level
binding", e.g. a reference to a reference (assuming that the
terminology is correct by saying that all Python variables are
references). This is where I see the hole in Python, that there
actually is a class of object that is different from everything
else; you can bind to anything at all in the language...except
references.
-tom!
Roy Smith
> Ah - but this is also Pythons fault. It has arbitrarily redefined the
> meaning of a mathematics operator - presumably simply because C, C++,
> Java, Basic and others have done this in the past.
> Personally, though, I'd go with Pascal on this one.
Most languages have two "equals" operators, one means assignment, the
other means test for equality. Depending on the language, these are
spelled "=" and "==" or ":=" and "=" (or a few other variations).
The wierd thing is that I don't think either one really matches what a
mathematician means when he writes "=". In math, "=" is neither an
assignment or a test: it's an assertion.
Adam Ruth
I started to write a response, and then I realized that it would be
pointless. We're just gonna have to be happy in each other's ignorance.
Our skulls are too thick for each other's arguments.
I'm gonna stick with the idea that we're both wrong.
Even-though-your-more-wrong-erly yours,
Adam Ruth
Stephen Horne
>Stephen Horne <intent...@blank.co.uk> wrote in message news:<7dc4hvslh7a8fk9p6...@4ax.com>...
>> On 13 Jul 2003 21:03:59 -0700, ow...@hotmail.com (Adam Ruth) wrote:
>>
>
>> C++ has precisely one type of variable. That variable is a placeholder
>> for a value of a datatype which is specified in the declaration. The
>> datatype may be a pointer datatype, but so what? Pointer datatypes are
>> not treated any differently than other datatype except that they, like
>> all datatypes, they have their own set of functionality.
>
>Granted. Pointers are no different than other data types, but they
>are typically used for operations that are semantically very different
>than other datatypes are used for. In that sense, they are, at a high
>level, different data types.
A different data type is nothing special. Mathematics has the concept
of abstract data types. That's the point. There's no need to invent
different kinds of variable, only different types of data. The fact
that C and C++ variables are restricted in the set of values they can
bind to (ie those with a particular data type) is not a problem.
>It's like how c programming is taught has having pass by reference and
>pass by value, when there is only pass by value at the implementation
>level. Pass by reference is a concept added on top of the language.
No. C programmers always pass by value. Sometimes that value is a
pointer.
Pascal programmers sometimes pass by reference. They do so explicitly,
however - using the keyword 'var'. There is a big difference between
*choosing* to have variables bound to an object, and having it happen
automatically whether you want it or not.
>> C++ references are tellingly also called self-dereferencing pointers.
>> They are not a distinct concept - they are syntactic sugar. I suspect
>> they mainly arise out of the modern desire to disguise pointers and
>> fantasize that they're not there, though they certainly work very well
>> in certain contexts.
>
>Syntactic sugar or no, they still behave differently than other
>datatypes and are therefore not consistent... IMHO.
They behave exactly like pointers. You just use different notation to
get that behaviour. That is what makes them syntactic sugar.
>> Funny thing. When I use algebra, the variables I define don't end up
>> referring to different values, functions or whatever unless I
>> explicitly redefine them. When I write a definition on one piece of
>> paper, the things I wrote earlier on another sheet don't change.
>>
>> Seems to me that the math equivalent of assignment (defining named
>> things) works very much like the 'static language definitions' as you
>> put it.
>
>The devil is in the details. Math assignment is static assignment is
>dynamic assignment. They really are all the same thing at a high
>level, but it's the implementation and the subtleties that make them
>vary.
Not true. Maths has the concept of an algorithm, and it has the
concept of assignment.
In maths, a variable binds to a value. In Python, a variable binds to
an object in a way that does no correctly implement binding to a value
unless the object happens to be immutable.
Mark 'Kamikaze' Hughes
> On Sun, 13 Jul 2003 22:42:21 GMT, "Bryan" <bel...@yahoo.com> wrote:
>>i just referenced an immutable object via a "pointer" and i __did_not__
>>stuff it into a mutable object as you say.
>>a and b "point" to the same object.
> Technically, but what is the point? You can't do pointer-style things
> with it. You can't change the object in any way without changing the
> id, and you can't use the mechanism to, for instance, allow 'var'
> parameters.
> In short, you are showing evidence of the use of pointers internally
> within Python, but that is not the same as providing pointer-like
> semantics.
It looks like you've almost reached understanding, though you've been
given a bad start with a very idiosyncratic and monolingual
understanding of what "computer science" teaches--that was certainly not
in the classes I took, but apparently yours were different.
Python variables are directly equivalent to pointers in C, with some
crippling limitations by C standards. They only ever point to valid
objects that are dynamically allocated on the heap, never to anything
else. Even None is an object. Those objects include what Python calls
"integers", which are really object wrappers around a C int variable.
Assignment only allows you to change a variable to point to some other
valid object.
At no point in Python do you ever have direct access to a C value
variable, or to a reference to a variable. They just don't exist. So
there are some things you can do in C that you can't do the same way in
Python. And that's *by intention*. It isn't a bad thing, it was done
for good design reasons, reasons well-tested by previous languages like
Lisp.
If you really must do a "pointer-style thing" in Python, you can do
this, which is semantically equivalent to the C version:
>>> def incr(pointerToVar):
... pointerToVar[0] += 1
...
>>> a = [0]
>>> b = a
>>> incr(a)
>>> a[0]
1
>>> b[0]
1
>>> incr(a)
>>> a[0]
2
>>> b[0]
2
I hope no Pythonista would actually *do* that, but it's working code,
and the foo[0] is similar to explicit referencing and dereferencing of
pointers.
Since every operation in Python operates on pointers, there's no use
in having a special syntax for it. You don't need all the * and & and
-> line noise.
Stop trying to make Python into C/C++, and you'll be happier with it.
Or stop using Python, if you really don't like the design philosophy.
There are plenty of Algol-derived languages out there. PHP and
especially Perl are more C-like in their internal logic, and you might
find them more pleasant.
--
<a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"The computer is incredibly fast, accurate, and stupid.
Man is unbelievably slow, inaccurate, and brilliant.
The marriage of the two is a force beyond calculation." -Leo Cherne
Adam Ruth
So, then, what makes a different variable type? If not different syntax
and semantics, what would be the signature?
Let me see if I understand what you're saying. Even though C++
variables can have different syntax and semantics (depending sometimes
on context), the're all really the same internally and are therefore
perfectly consistent and usable. Python variables, on the other hand (
mutable and immutable) have somewhat different semantics (a debatable
point) but identical implementaions (indeed, indistinguishable), but are
somehow invalid?
You say that there is one gold standard definition for what a variable
is. Yet even though C++ deals with variables in three semantically
distinct way, they are somehow all valid within that definition?
Adam Ruth
Stephen Horne
>Stephen Horne <intent...@blank.co.uk> wrote:
>> Ah - but this is also Pythons fault. It has arbitrarily redefined the
>> meaning of a mathematics operator - presumably simply because C, C++,
>> Java, Basic and others have done this in the past.
>> [...]
>> Personally, though, I'd go with Pascal on this one.
>
>Most languages have two "equals" operators, one means assignment, the
>other means test for equality. Depending on the language, these are
>spelled "=" and "==" or ":=" and "=" (or a few other variations).
That's why I referred to Pascal.
>The wierd thing is that I don't think either one really matches what a
>mathematician means when he writes "=". In math, "=" is neither an
>assignment or a test: it's an assertion.
99 out of 100 times, true. But not always - especially in discrete
maths.
For example, if I write...
[forall] x . f(x) = y
It's hard to interpret that '=' as an assertion. Similar applies to a
range of maths operators.
Basically, the way I rationalise it is that when I see what looks like
a boolean expression standing on its own, I take it as an assertion
that that expression evaluates to true. I'm not saying that has any
mathematical validity, but it does seem to cover all cases.
So if I see...
x = 1
I take it as an assertion that the boolean test 'x = 1' evaluates to
true, and from that I can infer the value bound to x (or, in other
cases, infer some properties of that value).
Or rather I don't - but that's the rationalisation for what I do do.
Tom Plunket
> As others have explained, you just return the value.
> It's just a different point of view. Instead of
> change (v)
> you have:
> v = change (v)
Ahh, now I remember what I wanted to do, and you're right- it's
more verbose and didn't really seem to add anything. I wanted to
in-place clamp integers in a range, or maybe they were even
member variables;
def Color:
def SomeOperation():
# modify self.r, g, b
self.r = clamp(self.r, 0, 255)
self.g = clamp(self.g, 0, 255)
self.b = clamp(self.b, 0, 255)
...that was just more typing than I wanted at that point,
although I'm sure that the keystrokes I burned were more than
made up for by Python's allowing me to do:
def clamp(val, lowVal, highVal):
if (lowVal <= val <= highVal):
return val
:)
-tom!
Tim Peters
> ...
> C++ references are tellingly also called self-dereferencing pointers.
> They are not a distinct concept - they are syntactic sugar. I suspect
> they mainly arise out of the modern desire to disguise pointers and
> fantasize that they're not there, though they certainly work very well
> in certain contexts.
You don't have to guess -- Stroustrup's "The Design and Evolution of C++"
explains his design rationales in detail. Section 3.7 is devoted to C++
references. The short course is that references were thought necessary to
support usable operator overloading; see the book for concrete examples; a
later "What Could and Should Have Been Different?" section explains why C++
references are a wart, but a pragmatically desirable wart.
About "modern desire", the earliest computer languages didn't expose
pointers at all (Lisp, COBOL, and Fortran are the prime early examples of
this). Language exposure of machine addresses came in with lower-level
systems-programming languages (of which C is the prime example today).
sism...@hebmex.com
> Sent: Lunes, 14 de Julio de 2003 06:15 p.m.
>
> What I specifically want is a way to have a "second-level
> binding", e.g. a reference to a reference (assuming that the
> terminology is correct by saying that all Python variables are
> references). This is where I see the hole in Python, that there
> actually is a class of object that is different from everything
> else; you can bind to anything at all in the language...except
> references.
>
> -tom!
>
What for? What would be the practical use for such a thing?
To rebind a name remotely? It's not very consistent with the
rest of the language.
To have functions which rebind a name in a hidden way seems
to me like a veritable fountain of possible bugs. Explicit
is better than implicit and all that.
Those operations might have sense in a language where a variable
name is directly related to a storage location, a memory address;
but Python is not like that, and trying to program in that style
in Python is quite painful.
I find Python to be quite clear and powerful, and never have needed
such functionality in all the different type of projects I've done,
small-to-medium, but varied: SQL parsers, XML parsers, email
messaging systems, GUI frontends, RPC servers and clients,
text-formatting utilities, etc... I've never found a single
instance where I would have needed such that.
A binding can't also be a container, that's a job for an object,
if you truly need to "pull" data from a code segment, then you
should use a dictionary, or a list, or an instance of some
other mutable class, heck, even an empty class:
class Inbox: pass
IB = Inbox()
def fn(obj):
obj.value = 25
obj.message = "Here's a value for you"
... etc
Even a scheme as simpleminded as this is infinitely better than
having functions rebind your variable names to other objects,
even if they're the same class.
This, at least, is my opinion on the subject.
-gustavo
Advertencia:La informacion contenida en este mensaje es confidencial y
restringida, por lo tanto esta destinada unicamente para el uso de la
persona arriba indicada, se le notifica que esta prohibida la difusion de
este mensaje. Si ha recibido este mensaje por error, o si hay problemas en
la transmision, favor de comunicarse con el remitente. Gracias.
Tom Plunket
> What I specifically want is a way to have a "second-level
> binding", e.g. a reference to a reference (assuming that the
> terminology is correct by saying that all Python variables are
> references). This is where I see the hole in Python, that there
> actually is a class of object that is different from everything
> else; you can bind to anything at all in the language...except
> references.
I just realized that this would be silly, complex, etc.
Say you could (as another poster mentioned) do something like
this:
>>> a = 5
>>> b = ref_to(5)
>>> a = 3
>>> print b
3
This would require a new built-in probably, and any time you
(accidentally?) did 'b = c', then you'd lose that reference,
but... hmm, I wonder if it'd be handy.
Probably not enough to justify the expense. :)
-tom!
Donald 'Paddy' McCarthy
> I want to do something along the lines of the following C++ code:
>
> void change(int& i)
> {
> i++;
> }
>
>
> Is there any way to do references like this in Python? It'd be
> like this, though clearly this doesn't change the thing I want it
> to change:
>
> def change(val):
> val += 1
>
> v = 0
> change(v)
>
> # v == 1 now.
>
> thanks.
>
> I can see that maybe if I passed in a list of one integer I could
> change that, but that seems circuitous to me.
>
>
> -tom!
Hi Tom,
Welcome to Python.
I guess the first thing to remember is that Python is not any other
language. It has its own ways of doing things, and its own nomenclature,
that while in some cases may use the same term or phrase as another
language, or symbols used in Mathematics - It puts a Python meaning on them.
Python does things its own way. I found that I got much more productive
when I learnt enough to let go of the C idioms in my case that I had
initially. I guess I was helped in that I was a good programmer in C,
didn't like C++ or Java and was looking for another language that would
be more productive. I could do the C pointer wizardry but in no way did
I think that it was ideal. It sometimes helps to have an understanding
of what Python is doing 'under the hood', but I don't solve my Python
problems by translating solutions expressed in C pointer terms.
Other mails have gone into the minutia of trying to answer the question.
I would just like to add that if you look up scope rules or the global
keyword you will find ways of accessing variables defined outside of a
function definition.