unexpected behavior: did i create a pointer?
gu
after two days debugging my code, i've come to the point that the
problem was caused by an unexpected behaviour of python. or by lack of
some information about the program, of course! i've stripped down the
code to reproduce the problem:
<code>
a = {}
for x in range(10):
for y in range(10):
a[x,y] = "0"
copyOfA = a
def functionA(x,y):
print a[x,y],
copyOfA[x,y] = "*"
print a[x,y],copyOfA[x,y]
for x in range(10):
for y in range(10):
functionA(x,y)
</code>
now, in the second "for" cycle and in functionA() i only 'touch' copyOfA
(altering it). as i don't touch the variable "a", i expect it not to be
affected by any change, but copyOfA acts like a pointer to a and
altering copyOfA's values result in altering the values of "a", so the
result that i expect is:
0 0 *
0 0 *
0 0 *
0 0 *
[..]
but i get:
0 * *
0 * *
0 * *
0 * *
[..]
what's going on?
thanks in advance.
Gabriel Genellina
> after two days debugging my code, i've come to the point that the
> problem was caused by an unexpected behaviour of python. or by lack of
> some information about the program, of course! i've stripped down the
> code to reproduce the problem:
>
> <code>
> a = {}
>
> for x in range(10):
> for y in range(10):
> a[x,y] = "0"
>
> copyOfA = a
copyOfA is *NOT* a copy - it's just another name pointing to the SAME
object as a.
Python will never copy anything unless told explicitely.
Read this <http://effbot.org/zone/python-objects.htm>
--
Gabriel Genellina
Roel Schroeven
> copyOfA = a
>
> now, in the second "for" cycle and in functionA() i only 'touch' copyOfA
> (altering it).
copyOfA isn't a copy of a; it's a different name bound to the same
object as a. You can verify that: id(a) and id(copyOfA) will return the
same value.
To make a copy of a (assuming a is a dict), you can do:
copyOfA = dict(a)
or
copyOfA = a.copy()
or more generally
import copy
copyOfA = copy.copy(a)
Cheers,
Roel
--
If I have been able to see further, it was only because I stood
on the shoulders of giants. -- Isaac Newton
Roel Schroeven
Wildemar Wildenburger
> hi to all!
> after two days debugging my code, i've come to the point that the
> problem was caused by an unexpected behaviour of python. or by lack of
> some information about the program, of course! i've stripped down the
> code to reproduce the problem:
>
Yes, basically you *created* a pointer. That's all that python has:
pointers.
When saying
>>> a = AnyOldObject()
>>> b = a
then 'a' and 'b' are different /names/ for the /very same/ object (try
"a is b", or "id(a)==id(b)").
This is really a FAQ (once a week or so?), but for the life of me I
can't find the right words for a google query.
TO THE TROOP: What keywords would you attach to that question?
/W
Steven D'Aprano
> gu wrote:
>> hi to all!
>> after two days debugging my code, i've come to the point that the
>> problem was caused by an unexpected behaviour of python. or by lack of
>> some information about the program, of course! i've stripped down the
>> code to reproduce the problem:
>>
>> [snip FAQ]
>
> Yes, basically you *created* a pointer. That's all that python has:
> pointers.
No, you are confusing the underlying C implementation with Python. Python
doesn't have any pointers. CPython is implemented with pointers. PyPy,
being written entirely in Python, is implemented with Python objects like
lists and dicts. Jython, being implemented in Java, probably isn't
implemented with pointers either -- although of course the underlying
Java compiler might be. IronPython and Python for .Net, I have no idea
how they work. Could be magic for all I know. (Probably necromancy.)
Naturally, regardless of whether you are using CPython, IronPython, PyPy
or some other variety of Python, the objects available to you include
ints, floats, strings, lists, dicts, sets and classes... but not pointers.
Nor does it include "peek" and "poke" commands for reading and writing
into random memory locations. Python is not C, and it is not Basic, nor
is it Forth or Lisp or assembler, and you shouldn't hammer the round peg
of Python objects into the square hole of C pointers.
--
Steven.
Marc 'BlackJack' Rintsch
> Nor does it include "peek" and "poke" commands for reading and writing
> into random memory locations.
I guess `ctypes` offers tools to write `peek()` and `poke()`. :-)
Ciao,
Marc 'BlackJack' Rintsch
Dustan
> hi to all!
Hi!
Welcome to Python! You might want to look at the documentation:
http://docs.python.org/
And the tutorials:
http://docs.python.org/tut/tut.html
Leaping into python from another language without looking at the above
documentation is not wise, since python has both a very different
structure and a somewhat different philosophy from other languages.
Peter Otten
> Python doesn't have any pointers.
Thinking of python variables or "names" as pointers should
get you a long way when trying to understand python's behaviour.
As long as you keep in mind that python doesn't have pointers to pointers,
and no pointer arithmetic either...
Peter
Grant Edwards
> Am Fri, 07 Sep 2007 10:40:47 +0000 schrieb Steven D'Aprano:
>
>> Python doesn't have any pointers.
>
> Thinking of python variables or "names" as pointers should
> get you a long way when trying to understand python's behaviour.
But thinking of them as names bound to objects will get you
further (and get you there faster). ;)
> As long as you keep in mind that python doesn't have pointers
> to pointers, and no pointer arithmetic either...
--
Grant Edwards grante Yow! Hello... IRON
at CURTAIN? Send over a
visi.com SAUSAGE PIZZA! World War
III? No thanks!
Wildemar Wildenburger
> On Fri, 07 Sep 2007 11:46:38 +0200, Wildemar Wildenburger wrote:
>
>> gu wrote:
>>> hi to all!
>>> after two days debugging my code, i've come to the point that the
>>> problem was caused by an unexpected behaviour of python. or by lack of
>>> some information about the program, of course! i've stripped down the
>>> code to reproduce the problem:
>>>
>>> [snip FAQ]
>> Yes, basically you *created* a pointer. That's all that python has:
>> pointers.
>
> No, you are confusing the underlying C implementation with Python.
I just thought I'd go along with the analogy the OP created as that was
his mindset and it would make things easier to follow if I didn't try to
forcibly change that.
Please note that I had intended for the word 'created' to be in quotes
rather than asterisks. I bit myself after I read my mistake online but
then thought "Who cares?". I should have, maybe :).
And yes, I will admit that going along with that analogy isn't the best
way to explain it. Grant Edwards in reply to Peter Otten makes it much
clearer, I guess.
/W
Steve Holden
error in vocabulary. gu said """Yes, basically you *created* a pointer.
That's all that python has: pointers.""" and you took issue with that,
ultimately saying """the objects available to you include ints, floats,
You are both right, and you are also both wrong.
Python does indeed provide a rich set of strongly typed object classes,
and so clearly to say "Python only has pointers" is strictly an error.
However, this overlooks the fact that *all* names in Python, and *all*
items in container objects (lists, tuples, dicts ...) hold references to
objects.
I mention this because it *is* significant to the semantics of
assignment. From the point of view of a C programmer Python might
actually look quite like a language in which all data objects had to be
malloc'd, and the only variables were pointers to. Assignment (binding)
in Python creates copies of references to objects, not copies of the
objects themselves.
I have deliberately omitted discussion of the automatic dereferencing
that takes place in Python, thereby allowing us to treat names as though
they contained objects, but the fact remains that names *don't* contain
objects, they contain references to objects. If you want to regard a
reference and a pointer as two different things then I guess that's your
nit to pick. But I don't think the original assertions quite justified
your scathing remarks about "peek" and "poke".
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
Steve Holden
> On 2007-09-07, Peter Otten <__pet...@web.de> wrote:
>> Am Fri, 07 Sep 2007 10:40:47 +0000 schrieb Steven D'Aprano:
>>
>>> Python doesn't have any pointers.
>> Thinking of python variables or "names" as pointers should
>> get you a long way when trying to understand python's behaviour.
>
> But thinking of them as names bound to objects will get you
> further (and get you there faster). ;)
>
Since it's only a matter of time before someone brings up the "post-It"
analogy, let me cavil in advance about it. The thing I don't like about
that particular pedagogic mechanism is that it always "attaches" the
names to the objects.
To me that blurs the fact that the names live strictly inside
namespaces, and are destroyed (along with the references they make to
other objects) when the namespace goes out of scope or otherwise ceases
to exist (instance destruction being the obvious one).
I only mention this because I too can be a pedantic little bugger when I
want to be. Have a nice day.
Steven D'Aprano
> I just thought I'd go along with the analogy the OP created as that was
> his mindset and it would make things easier to follow if I didn't try to
> forcibly change that.
My reaction to somebody trying to reason with the wrong analogy is to
teach them the right analogy, not to tell them they got it right when
they actually got it wrong.
"My car won't start -- I must not have stirred the gasoline enough before
baking it."
"Yes, that's right. It's very important to stir the gasoline fully so
that all the ingredients are fully mixed."
--
Steven.
Peter Otten
I think Wildemar is too defensive. The pointer "analogy" is a good first
approximation, not cargo cult.
Peter
Peter Otten
> On 2007-09-07, Peter Otten <__pet...@web.de> wrote:
>> Am Fri, 07 Sep 2007 10:40:47 +0000 schrieb Steven D'Aprano:
>>
>>> Python doesn't have any pointers.
>>
>> Thinking of python variables or "names" as pointers should
>> get you a long way when trying to understand python's behaviour.
>
> But thinking of them as names bound to objects will get you
> further (and get you there faster). ;)
Understanding a new system in terms of one I already know works for me.
When terminology and the system it describes make a perfect fit that is a
strong indication that you have reached a dead end.
Peter
Steven D'Aprano
Ways that Python objects are not like C pointers:
(1) You don't have to manage memory yourself.
(2) You don't have typecasts. You can't change the type of the object you
point to.
(3) Python makes no promises about the memory location of objects.
(4) No pointer arithmetic.
(5) No pointers to pointers, and for old-school Mac programmers, no
handles.
(6) No dangling pointers. Ever.
(7) There's no null pointer. None is an object, just like everything else.
(8) You can't crash your computer by writing the wrong thing to the wrong
pointer. You're unlikely even to crash your Python session.
Ways that Python objects are like pointers:
(1) ... um...
Oh yeah, if you bind the _same_ object to two different names, _and_ the
object is mutable (but not if it is immutable), mutating the object via
one name will have the same effect on the object -- the same object,
naturally -- bound to the other name.
You know, maybe because I came to Python with no C experience, I never
had trouble with the "unexpected behaviour" that so confused the original
poster. It's just obvious.
--
Steven.
Wildemar Wildenburger
> On Fri, 07 Sep 2007 12:19:12 -0700, Steve Holden <st...@holdenweb.com>
> declaimed the following in comp.lang.python:
>
>> Since it's only a matter of time before someone brings up the "post-It"
>> analogy, let me cavil in advance about it. The thing I don't like about
>> that particular pedagogic mechanism is that it always "attaches" the
>> names to the objects.
>>
> "post-it notes" -- attached to the ends of strings that then run to the
> "physical object" at the other end.
This is getting clearer by the minute. ;)
/W
Arnaud Delobelle
cybersource.com.au> wrote:
[...]
> Ways that Python objects are like pointers:
>
> (1) ... um...
>
> Oh yeah, if you bind the _same_ object to two different names, _and_ the
> object is mutable (but not if it is immutable), mutating the object via
> one name will have the same effect on the object -- the same object,
> naturally -- bound to the other name.
Well one of the main uses of pointers in C is as things that *point
to* objects. And AFAIK that's exactly what a name is in Python. In
fact I think to say that a name points to (or refers to) an object is
less misleading that to say it is bound to. Binding implies some sort
of symmetry but when I write:
a = "Am I bound?"
The name a knows it's refering to the string object, whereas the
string has no idea who refers to it (well an implementation might want
to store this information, but it is inaccessible).
> You know, maybe because I came to Python with no C experience, I never
> had trouble with the "unexpected behaviour" that so confused the original
> poster. It's just obvious.
The funny thing is that if the OP had thought of both 'a' and
'copyOfA' as C-like pointers then he wouldn't have been confused :)
--
Arnaud
Steven D'Aprano
>> You know, maybe because I came to Python with no C experience, I never
>> had trouble with the "unexpected behaviour" that so confused the
>> original poster. It's just obvious.
>
> The funny thing is that if the OP had thought of both 'a' and 'copyOfA'
> as C-like pointers then he wouldn't have been confused :)
You're almost certainly wrong. He would have written to ask why this
doesn't do what he expects:
>>> x = 3
>>> y = x # x and y are both pointers to the same value
>>> x += 1
>>> print x == y # of course, they are pointers to the same value
False
Or why lower_list() works as expected, but lower_string() doesn't:
>>> def lower_list(L):
... for i, x in enumerate(L):
... L[i] = x.lower()
...
>>> s = ['STRING']
>>> lower_list(s)
>>> print s == ['string']
True
>>>
>>> def lower_string(s):
... s = s.lower()
...
>>> s = "STRING"
>>> lower_string(s)
>>> print s == "string"
False
The "names in Python are pointers" analogy only gives you the right
answer half the time. The "names in Python are names" analogy gives you
the right answer ALL THE TIME, no exceptions.
--
Steven.
Arnaud Delobelle
cybersource.com.au> wrote:
> On Sun, 09 Sep 2007 02:30:00 -0700, Arnaud Delobelle wrote:
> >> You know, maybe because I came to Python with no C experience, I never
> >> had trouble with the "unexpected behaviour" that so confused the
> >> original poster. It's just obvious.
>
> > The funny thing is that if the OP had thought of both 'a' and 'copyOfA'
> > as C-like pointers then he wouldn't have been confused :)
>
> You're almost certainly wrong. He would have written to ask why this
> doesn't do what he expects:
>
> >>> x = 3
> >>> y = x # x and y are both pointers to the same value
> >>> x += 1
This means that x now points to the value of x + 1, i.e. an int object
with value 4.
> >>> print x == y # of course, they are pointers to the same value
Of course not, now x points to 4 and y points to 3 !
> False
>
> Or why lower_list() works as expected, but lower_string() doesn't:
>
> >>> def lower_list(L):
>
> ... for i, x in enumerate(L):
> ... L[i] = x.lower()
> ...>>> s = ['STRING']
> >>> lower_list(s)
> >>> print s == ['string']
> True
>
> >>> def lower_string(s):
>
> ... s = s.lower()
> ...>>> s = "STRING"
> >>> lower_string(s)
Let's see what happens here: when lower_string(s) is called, the 's'
which is local to lower_string is made to point to the same object as
the global s (i.e. the string object with value "STRING"). In the
body of the function, the statement s=s.lower() makes the local 's'
point to a new string object returned s.lower(). Of course this has
not effect on what object the global 's' points to.
> >>> print s == "string"
Obviously not, since s still points to the string object with value
"STRING"
> False
>
> The "names in Python are pointers" analogy only gives you the right
> answer half the time.
They give *you* the right answer only half the time ;)
They give me the right answer all the time. Not that I usually think
of names as pointers. But if I choose to do so, I can still get it
right. And if it works for me consistently, there must be some
validity in it, no?
What I think is a more dangerous misconception is to think that the
assignement operator (=) has the same meaning in C and python.
--
Arnaud
Alex Martelli
...
> > >>> def lower_list(L):
> >
> > ... for i, x in enumerate(L):
> > ... L[i] = x.lower()
> > ...>>> s = ['STRING']
> > >>> lower_list(s)
> > >>> print s == ['string']
> > True
> >
> > >>> def lower_string(s):
> >
> > ... s = s.lower()
> > ...>>> s = "STRING"
> > >>> lower_string(s)
>
> Let's see what happens here: when lower_string(s) is called, the 's'
> which is local to lower_string is made to point to the same object as
> the global s (i.e. the string object with value "STRING"). In the
> body of the function, the statement s=s.lower() makes the local 's'
> point to a new string object returned s.lower(). Of course this has
> not effect on what object the global 's' points to.
Yep, the analogy with C pointers would work fine here:
void lower_string(char* s) {
s = <whatever>
}
would fail to have the intended effect in C just as its equivalent does
in Python (in both Python and C, rebinding the local name s has no
effect on the caller of lower_string). Add an indirection:
void lower_list(item* L) {
...
L[i] = <something>
}
this indirection (via indexing) *does* modify the memory area (visible
by the caller) to which L points.
The difference between "name=something" and "name[i]=something" is so
*HUGE* in C (and in Python) that anybody who doesn't grok that
difference just doesn't know or understand any C (nor any Python).
> What I think is a more dangerous misconception is to think that the
> assignement operator (=) has the same meaning in C and python.
I've seen the prevalence of that particular misconception drop
dramatically over the years, as a growing fraction of the people who
come to Python after some previous programming experience become more
and more likely to have been exposed to *Java*, where assignment
semantics are very close to Python (despite Java's unfortunate
complication with "unboxed" elementary scalar types, in practice a vast
majority of occurrences of "a=b" in Java have just the same semantics as
they do in Python); teaching Python semantics to people with Java
exposure is trivially easy (moving from "ALMOST every variable is an
implicit reference -- excepting int and float ones" to "EVERY variable
is an implicit reference"...).
Alex
neoedmund
no language act like want you tought, or the assignment operation will
be too expensive.
Peter Otten
> Ways that Python objects are not like C pointers:
>
> (1) You don't have to manage memory yourself.
>
> (2) You don't have typecasts. You can't change the type of the object you
> point to.
>
> (3) Python makes no promises about the memory location of objects.
>
> (4) No pointer arithmetic.
>
> (5) No pointers to pointers, and for old-school Mac programmers, no
> handles.
>
> (6) No dangling pointers. Ever.
>
> (7) There's no null pointer. None is an object, just like everything else.
>
> (8) You can't crash your computer by writing the wrong thing to the wrong
> pointer. You're unlikely even to crash your Python session.
>
>
>
> Ways that Python objects are like pointers:
>
> (1) ... um...
>
> Oh yeah, if you bind the _same_ object to two different names, _and_ the
> object is mutable (but not if it is immutable), mutating the object via
> one name will have the same effect on the object -- the same object,
> naturally -- bound to the other name.
Had you put it that way in the first place I would have stayed in in my
hole ;)
Peter
Bruno Desthuilliers
>> now, in the second "for" cycle and in functionA() i only 'touch' copyOfA
>> (altering it). as i don't touch the variable "a", i expect it not to be
>> affected by any change, but copyOfA acts like a pointer to a and
>> altering copyOfA's values result in altering the values of "a", so the
>> result that i expect is:
> no language act like want you tought, or the assignment operation will
> be too expensive.
IIRC, php4 was doing copies for arrays...
Gabriel Genellina
<bruno.42.de...@wtf.websiteburo.oops.com> escribi�:
Pascal does copy arrays and structs (records) too.
--
Gabriel Genellina