Is 'everything' a refrence or isn't it?
KraftDiner
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:
lst = [1,2,3]
for i in range(len(lst)):
if lst[i] == 2:
lst[i]=4
print lst
Have I misunderstood something?
Christian Tismer
> I was under the assumption that everything in python was a refrence...
This is true.
> so if I code this:
> lst = [1,2,3]
lst is a reference to a list that holds references to 1, 2, and 3
> for i in lst:
i is a reference to every element in the list, one ofter the other.
But this is not a ref to lst any longer, the list gets dereferenced
before getting at its contents, and only references to that are
returned.
> if i==2:
> i = 4
You now have i as a reference to 4, during this cycle.
ciao - chris
--
Christian Tismer :^) <mailto:tis...@stackless.com>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
Grant Edwards
> I was under the assumption that everything in python was a refrence...
It is.
> so if I code this:
> lst = [1,2,3]
> for i in lst:
> if i==2:
> i = 4
> print lst
>
> I though the contents of lst would be modified..
Nope. "i = 4" doesn't modify the object. It changes "i" to
point to a different object (one that is an integer value 4).
> (After reading that
> 'everything' is a refrence.)
> so it seems that in order to do this I need to code it like:
>
> lst = [1,2,3]
> for i in range(len(lst)):
> if lst[i] == 2:
> lst[i]=4
> print lst
>
> Have I misunderstood something?
Sort of.
You've misunderstood what the statement "i = 4" means.
--
Grant Edwards grante Yow! I'LL get it!! It's
at probably a FEW of my
visi.com ITALIAN GIRL-FRIENDS!!
Duncan Booth
> I was under the assumption that everything in python was a refrence...
It is, although it is better to think in terms of names and bindings.
>
> so if I code this:
> lst = [1,2,3]
> for i in lst:
> if i==2:
> i = 4
> print lst
>
> I though the contents of lst would be modified.. (After reading that
> 'everything' is a refrence.)
During execution of this code, the name 'i' is bound to 1, 2, 3 and 4. the
list elements are bound to 1, 2, and 3. Rebinding 'i' such that instead of
referencing 2 it now references 4 doesn't affect the fact that the second
element of the list references 2.
> so it seems that in order to do this I need to code it like:
>
> lst = [1,2,3]
> for i in range(len(lst)):
> if lst[i] == 2:
> lst[i]=4
> print lst
>
> Have I misunderstood something?
Evidently.
Scott David Daniels
> Have I misunderstood something?
yes.
There is a reason people talk about names and bindings.
--
-Scott David Daniels
scott....@acm.org
Simon Brunning
> I was under the assumption that everything in python was a refrence...
> Have I misunderstood something?
Yup. The concept if a reference is, I find, sometimes an unhelpful one
in Python. Reset your brain -
<http://effbot.org/zone/python-objects.htm>.
Armed with that, let's go through your code:
> lst = [1,2,3]
Here, you've created a list object, and bound it to the name "lst".
The list contains references to three integer objects. (OK I said
"reference" there didn't I? Well, I only said that it's *sometimes* an
unhelpful concept. ;-) I do tend to think of collections as refering
to objects. It's when dealing with you might call variables that it's
misleading - there it's better to think of names, objects, and
bindings.)
> for i in lst:
Here, you are iterating through the list that's currently bount to the
name "lst". You are binding each element of the list in turn to the
name "i".
> if i==2:
If the object to which the name "i" is currently bound compares equal to 2...
> i = 4
Rebind the name "i" to the integer object 4. Note that this has no
effect on the object that used to be bound to "i" at this point, nor
any effect on the list object.
> print lst
Print your old list.
Clear now?
--
Cheers,
Simon B,
si...@brunningonline.net,
http://www.brunningonline.net/simon/blog/
Claudio Grondi
And now into the details assuming, that your misunderstanding comes from
programming in C before evaluating Python:
If you write
i = 4
following happens:
an integer literal 4 gets into the scope of Python script as an object
of type integer and can be from now on reached using the identifier i .
So i is in this context a _kind of_ pointer/reference to the value
'4', but ... in Python sense not in C sense.
if you write
for i in lst:
then the identifier i 'points' to values in lst, but else as in C
where you can use a pointer to change the value it points to by
assigning to *i or where you can use a reference to change a value it
references, in Python you don't change a value of i because i is an
identifier and therefore has no value in sense of C - it is only a name
used to reach a value.
When you write within the 'for i in lst:' loop
i = 4
you just use the identifier i as a 'pointer' to another Python object
as it was pointing to before.
Let's consider a list.
The 'identifier'(i.e. construct) lst[i] pointing to a list element
behaves in Python like what you understand in C as a reference.
This is the main difference between the meaning of = assignment in
Python and in C and it usually results in problems with understanding it
when someone has programmed C and expects from Python to be like C.
The core of the trouble is probably different understanding of the word
'reference', which used in different contexts (e.g. Python and C) means
different things. Therefore as Scott David Daniels says
"There is a reason people talk about names and bindings."
Claudio
Steven D'Aprano
> I was under the assumption that everything in python was a refrence...
>
> so if I code this:
> lst = [1,2,3]
> for i in lst:
> if i==2:
> i = 4
> print lst
>
> I though the contents of lst would be modified.. (After reading that
> 'everything' is a refrence.)
See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.
It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.
It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.
In the above code, i is a name bound one at a time to the ints [1,2,3].
When you re-assign i to 4, that doesn't change the object 2 into the
object 4, because ints are immutable. Only the name i is rebound to a new
object 4. That doesn't change objects like lst which include 2 inside them.
See this for more detail:
http://effbot.org/zone/call-by-object.htm
--
Steven.
Steven D'Aprano
> KraftDiner wrote:
>> I was under the assumption that everything in python was a refrence...
>
> This is true.
No it is not.
>> so if I code this:
>> lst = [1,2,3]
>
> lst is a reference to a list that holds references to 1, 2, and 3
>
>> for i in lst:
>
> i is a reference to every element in the list, one ofter the other.
> But this is not a ref to lst any longer, the list gets dereferenced
> before getting at its contents, and only references to that are
> returned.
See what confusing, complicated hoops people have to jump through to
hammer the round peg of Python's behaviour into the square peg of "call by
reference"?
http://effbot.org/zone/call-by-object.htm
Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.
--
Steven.
Claudio Grondi
using pictures showing what happens when an assignment is processed
demonstrating what is what and how it is called. Probably a small nice
movie could be here the right medium to be handed over to Python newbies
coming from C to give some hints towards proper understanding. Replacing
one word with another is not sufficient to avoid confusion as the
concept is probably not always easy to grasp (I have still problems to
get the idea how it works 'inside' with this local and global
dictionaries and so on).
Lack of knowledge about a good reference was the reason why I decided to
give a reply here inspite of the fact, that so many other postings were
already there, but none (including mine) which covered all aspects, so
that I could say: "Wow! It is an excelent explanation and there is
nothing which must be added to it!".
Is there anywhere an ultimative enlightening explanation of the concept
of identifiers and variables in Python for use in case the next posting
of that kind hits comp.lang.python and it sure will as they are coming
in quite regular.
http://effbot.org/zone/call-by-object.htm is sure an attempt of a good
explanation, but in my eyes still nothing what immediately enlightens.
Claudio
Peter Hansen
> Python does not have references or pointers, except internally where
> Python coders can not get to them. It has names and objects. Keep thinking
> about "call by reference" and you just confuse yourself and others. Think
> about names and objects and it is simple and straight-forward.
I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.
What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation of
binding (i.e. usually what assignment does).
-Peter
Mike Meyer
> On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
>> I was under the assumption that everything in python was a refrence...
>> so if I code this:
>> lst = [1,2,3]
>> for i in lst:
>> if i==2:
>> i = 4
>> print lst
>> I though the contents of lst would be modified.. (After reading that
>> 'everything' is a refrence.)
> See, this confusion is precisely why I get the urge to slap people who
> describe Python as "call by reference". It isn't.
Except this doesn't have *anything at all* to do with python being (or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.
> It is "call by object" -- you pass around *objects*. Internally, this is
> quite fast, because the entire object doesn't need to be moved, only
> pointers to objects, but you don't get the behaviour of either call by
> reference or call by value.
No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference. If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.
<mike
--
Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Mike Meyer
> Steven D'Aprano wrote:
>> Python does not have references or pointers, except internally where
>> Python coders can not get to them. It has names and objects. Keep thinking
>> about "call by reference" and you just confuse yourself and others. Think
>> about names and objects and it is simple and straight-forward.
> I won't argue the point, but I would point out that the term "name" is
> insufficient for whatever it is that is stored inside a list.
Correct. What's stored in a list is a reference.
> What do you call it, if not a reference? The word "binding" isn't
> really appropriate here, as it is easily confused with the operation
> of binding (i.e. usually what assignment does).
To clarify, assignement statements in python are bindings, and don't
do assign values to a variables. Assignment in other languages do that
- which is why people get confused when exposed to what Python does.
Steven D'Aprano
> On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
>>I though the contents of lst would be modified.. (After reading that
>>'everything' is a refrence.)
>
> See, this confusion is precisely why I get the urge to slap people who
> describe Python as "call by reference". It isn't.
On reading back over my post, I realise that it might
sound like I was mad at KraftDiner. My apologies -- I'm
not, I feel (s)he is the victim of incorrect
information here, not the culprit.
After all, as a Python newbie, how is KraftDiner
supposed to know that when people say "Python is call
by reference", what they actually mean is "Um, except
that it doesn't behave like any call by reference
language you're likely to have used before, and
sometimes it behaves more like call by value"?
--
Steven.
Donn Cave
Steven D'Aprano <st...@REMOVETHIScyber.com.au> wrote:
> See what confusing, complicated hoops people have to jump through to
> hammer the round peg of Python's behaviour into the square peg of "call by
> reference"?
>
> http://effbot.org/zone/call-by-object.htm
>
> Python does not have references or pointers, except internally where
> Python coders can not get to them. It has names and objects. Keep thinking
> about "call by reference" and you just confuse yourself and others. Think
> about names and objects and it is simple and straight-forward.
I agree that it can be simple and straightforward. The particular
terms we use have nothing to do with it, though. Most of us need
a word or two to attach to the concept, because that's how our
brains work, but there are no natural words in the English language
that precisely express Python concepts, and we just have to make
do with one or another of several possible. It's silly to insist
that one particular word resolves all confusions, and another word
creates them. It's sort of unfortunate to let this crusade get to
the point where you're upbraiding people who know a great deal about
the matter because they don't use your favorite word.
As other followups have pointed out, the problem is simply not
understanding what "=" does. While in some obscure way this
may be related to "call by reference" etc., that's certainly not
a helpful digression. To explain what i = 4 does, we can say
creates a named reference "i" to 4
binds the name "i" to 4
etc.
Any of these is delightfully clear if you already understand it,
which is where most of the born again believers are coming from.
People who don't already understand what's being explained, need
to have a little patience and make sure to test their understanding
with a few experiments.
Donn Cave, do...@u.washington.edu
Steven D'Aprano
[snip]
>> See, this confusion is precisely why I get the urge to slap people who
>> describe Python as "call by reference". It isn't.
>>
>> It isn't "call by value" either -- Python never copies objects unless you
>> explicitly tell it to.
>>
>> It is "call by object" -- you pass around *objects*. Internally, this is
>> quite fast, because the entire object doesn't need to be moved, only
>> pointers to objects, but you don't get the behaviour of either call by
>> reference or call by value.
...
> It seems to be hard to explain it all in a straighforward way without
> using pictures showing what happens when an assignment is processed
> demonstrating what is what and how it is called.
That's no different from call by reference or call by
value. If you think back to when you were first
learning about pointers, that was confusing too.
> Probably a small nice
> movie could be here the right medium to be handed over to Python newbies
> coming from C to give some hints towards proper understanding. Replacing
> one word with another is not sufficient to avoid confusion as the
> concept is probably not always easy to grasp (I have still problems to
> get the idea how it works 'inside' with this local and global
> dictionaries and so on).
No, just saying "call by object" is not sufficient, but
it is vital. If you say "call by reference", the term
comes with a lot of mental baggage -- people think they
understand what call by reference means, how it
behaves, and as soon as they hear CBR they think they
understand how Python behaves. They think "Oh, Python
is CBR just like language Foo, so it must behave just
like language Foo, and this idiom works in language
Foo, so it must work in Python too."
And sometimes it does appear to work, which just makes
it all the more puzzling when it doesn't.
When they hear CBO the term is new and different and
doesn't come with any mental frame, so instead of
thinking they understand Python's behaviour, they think
"Call by what now? How does that work?" instead of
making incorrect assumptions.
--
Steven.
Steven D'Aprano
I just call it an item of the list. The fact that
CPython uses an array of pointers to objects for lists
is an implementation detail. Unfortunately, while
"reference" is a nice generic English word, in this
context it has too much mental baggage to safely use
around newbies. (Experience Pythonistas is another
story of course.)
--
Steven.
Steven D'Aprano
> Steven D'Aprano <st...@REMOVETHIScyber.com.au> writes:
>
>>On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
>>
>>>I was under the assumption that everything in python was a refrence...
>>>so if I code this:
>>>lst = [1,2,3]
>>>for i in lst:
>>> if i==2:
>>> i = 4
>>>print lst
>>>I though the contents of lst would be modified.. (After reading that
>>>'everything' is a refrence.)
>>
>>See, this confusion is precisely why I get the urge to slap people who
>>describe Python as "call by reference". It isn't.
>
>
> Except this doesn't have *anything at all* to do with python being (or
> not being) call by reference. This is a confusion about name binding
> vs. assignment to a variable. The proper people to slap around for
> this case are the ones who talk about assignment to a variable.
Mike, you are wrong. Not about talk about "assigning to
variables" being harmful, I agree with that. But this
confusion is *exactly* about the call by reference
misunderstanding. How do I know this?
Because the Original Poster said so! He said, to
paraphrase, "Hey, I thought Python was call by
reference, but I tried this, and it didn't work, what
gives???"
This is not an isolated case either.
You can't get any clearer than that. Talk about call by
reference leads to confusion and incorrect assumptions
about Python's behaviour. End of story.
>>It is "call by object" -- you pass around *objects*. Internally, this is
>>quite fast, because the entire object doesn't need to be moved, only
>>pointers to objects, but you don't get the behaviour of either call by
>>reference or call by value.
>
>
> No, you get *exactly* that behavior from call by reference when you
> start passing objects around by reference.
Who cares what Python does internally? It is strictly
irrelevant whether Python internally passes around
pointers, or copies values, or sometimes one or the other.
> If I declare a C object as
> "struct foo bar" and do the C "call-by-reference" hack of passing
> &bar, I get the exact same behavior I get when I pass an object
> referenced by bar to a Python subroutine.
It is not a question of whether Python's behaviour is
*sometimes* the same as call by reference. The question
whether Python's behaviour is *always* the same as CBR,
and it is not. Regardless of the reasons, Python does
not behave like CBR in other languages, and it is
harmful to imply that it does.
Likewise, you shouldn't argue that Python is obviously
call by value just because *sometimes* it has the same
behaviour as CBV.
I mean, come on! A whole bunch of people argue black
and blue that Python "obviously" is call by reference,
and another vocal bunch argue that it is "obviously"
call by value. Isn't that the tiniest hint that Python
-- not the underlying C/Java/Python/whatever
implementation, but Python the language --- does not
fall into either camp, but is doing something that
sometimes looks like one and sometimes like the other?
Just google on "Python is call by value" and "Python is
call by reference" -- including quotes -- to see this
argument come up time and time again, year after year
after year. And all because of some false dichotomy
that CBR and CBV are the only two valid descriptions of
a computer language.
--
Steven.
Steven D'Aprano
> Peter Hansen <pe...@engcorp.com> writes:
>
>
>>Steven D'Aprano wrote:
>>
>>>Python does not have references or pointers, except internally where
>>>Python coders can not get to them. It has names and objects. Keep thinking
>>>about "call by reference" and you just confuse yourself and others. Think
>>>about names and objects and it is simple and straight-forward.
>>
>>I won't argue the point, but I would point out that the term "name" is
>>insufficient for whatever it is that is stored inside a list.
>
>
> Correct. What's stored in a list is a reference.
Nonsense. What is stored in the list is an object.
Python doesn't have pointers. You are confusing the
underlying C implementation, which implements lists as
an array of pointers, with the high-level Python code,
which doesn't even know what a pointer is. It only has
objects.
Since we are discussing the perspective from a Python
programmer's point of view, it is irrelevant what the C
implementation does. If lists held references, you
could do this:
x = 5
L = [1, 2, x]
# the third item points to the same memory location
# as x; changing it will also change x
L[2] = 7
assert x == 7
which is the behaviour that "call by reference"
implies. But it doesn't work that way in high-level
Python code.
The *reason* it doesn't work is that the object 5 is
immutable. If you only ever passed mutable objects
around, you would think Python was call by reference.
And if you only ever passed immutable objects around,
you would think Python was call by value. But since
Python has both sorts of objects, it displays both
sorts of behaviour.
--
Steven.
Aahz
Steven D'Aprano <st...@REMOVEMEcyber.com.au> wrote:
>
>Nonsense. What is stored in the list is an object. Python doesn't have
>pointers. You are confusing the underlying C implementation, which
>implements lists as an array of pointers, with the high-level Python
>code, which doesn't even know what a pointer is. It only has objects.
Wrong. Python has objects and targets. Names are a subset of targets.
A target is anything that can contain a binding (or reference, whichever
term you prefer) to an object; lists are arrays of targets.
No matter how you slice it ;-), a list does not actually hold objects, it
only holds references to objects.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"19. A language that doesn't affect the way you think about programming,
is not worth knowing." --Alan Perlis
Mike Meyer
And he's right, and you're wrong. Look at the *code*. There isn't a
single call in it. He may have said "call by reference", but his
example code didn't have anything to do with call by rereference, and
he didn't demonstrate any confusion about the semantics of parameter
passing. He *did* demonstrate confusion about the semantics of the
assignment statement.
>>>It is "call by object" -- you pass around *objects*. Internally, this is
>>>quite fast, because the entire object doesn't need to be moved, only
>>>pointers to objects, but you don't get the behaviour of either call by
>>>reference or call by value.
>> No, you get *exactly* that behavior from call by reference when you
>> start passing objects around by reference.
> Who cares what Python does internally? It is strictly irrelevant
> whether Python internally passes around pointers, or copies values, or
> sometimes one or the other.
Who said anything about what Python does internally? I agree, it's
strictly irrelevant. What matters is the *behavior*.
>> If I declare a C object as
>> "struct foo bar" and do the C "call-by-reference" hack of passing
>> &bar, I get the exact same behavior I get when I pass an object
>> referenced by bar to a Python subroutine.
> It is not a question of whether Python's behaviour is *sometimes* the
> same as call by reference. The question whether Python's behaviour is
> *always* the same as CBR, and it is not.
You're wrong. Python's behavior is *always* the same as CBR, once you
identify what Python is passing a reference to.
> Likewise, you shouldn't argue that Python is obviously call by value
> just because *sometimes* it has the same behaviour as CBV.
Actually, it's either always the same as CBV, or never the same as
CBV. It's always the same as CBV because you always pass a value - and
the value is always a reference. But every calling mechanism requires
that you always pass a value. It's what the value is that determines
the calling mechanism. You pass a reference for CBR, and a name for
call by name.
> I mean, come on! A whole bunch of people argue black and blue that
> Python "obviously" is call by reference, and another vocal bunch argue
> that it is "obviously" call by value.
I don't think I've seen anyone arguing that Python is
call-by-value. There are people who argue that it's sometimes one and
soemtimes the other, but that's mostly because they have incorrectly
identified what Python is passing a reference to.
> Just google on "Python is call by value" and "Python is call by
> reference" -- including quotes -- to see this argument come up time
> and time again, year after year after year. And all because of some
> false dichotomy that CBR and CBV are the only two valid descriptions
> of a computer language.
I tried the google, and I don't see a lot of arguments. I see people
declaiming one way or another, and not very many of those. I don't
think it's an apparent dichotomy has much to do with it. I know there
are other things than CBV and CBR, and even understand Jensen's
device.
Shalabh Chaturvedi
> I was under the assumption that everything in python was a refrence...
For a good writeup on the 'name-binding' behavior others are talking about:
http://starship.python.net/crew/mwh/hacks/objectthink.html
Cheers,
Shalabh
Stuart D. Gathman
> I was under the assumption that everything in python was a refrence...
>
> so if I code this:
> lst = [1,2,3]
> for i in lst:
> if i==2:
> i = 4
> print lst
>
> I though the contents of lst would be modified.. (After reading that
> 'everything' is a refrence.)
> ...
> Have I misunderstood something?
It might help to do a translation to equivalent C:
int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };
int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
if (*i == *_i2)
i = &_i4;
}
for (_idx = 0; _idx < NLST; ++_idx)
printf("%d\n",*lst[_idx]);
--
Stuart D. Gathman <stu...@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Mike Meyer
>> Peter Hansen <pe...@engcorp.com> writes:
>>>Steven D'Aprano wrote:
>>>>Python does not have references or pointers, except internally where
>>>>Python coders can not get to them. It has names and objects. Keep thinking
>>>>about "call by reference" and you just confuse yourself and others. Think
>>>>about names and objects and it is simple and straight-forward.
>>>I won't argue the point, but I would point out that the term "name" is
>>>insufficient for whatever it is that is stored inside a list.
>> Correct. What's stored in a list is a reference.
> Nonsense. What is stored in the list is an object. Python doesn't have
> pointers. You are confusing the underlying C implementation, which
> implements lists as an array of pointers, with the high-level Python
> code, which doesn't even know what a pointer is. It only has objects.
I didn't say a single word about C, or pointers, or anything else to
do with the implementation.
> Since we are discussing the perspective from a Python
> programmer's point of view, it is irrelevant what the C implementation
> does. If lists held references, you could do this:
>
> x = 5
> L = [1, 2, x]
> # the third item points to the same memory location
> # as x; changing it will also change x
> L[2] = 7
> assert x == 7
>
> which is the behaviour that "call by reference" implies. But it
> doesn't work that way in high-level Python code.
Now you're talking nonsense. Call by reference is about *parameter
passing*, not about the behavior of assignment statements or
lists. How the above code behaves has *nothing* to do with how Python
passes parameters.
> The *reason* it doesn't work is that the object 5 is immutable.
The immutability of 5 has *nothing* to do with it. If you replace 5
and 7 with mutable objects, you get the exact same behavior:
x = [5]
L = [1, 2, x]
L[2] = [7]
assert x == set([7])
You seem to be suffering from the newbie confusion of thinking that a
Python assignment statement writes a value into the lvalue. It doesn't
- it makes the lvalue a reference to the object the right side
evaluates to ("binds a name to a value", except sometimes the lvalue
isn't a name). I'm pretty sure you know the difference.
The same thing applies to CBR/CBV. The immutability of the objects
being passed ain't got nothing to do with it. It's the fact that
assignment statements bind values instead of write them into a
variable that makes Python's CBR behavior sometimes act like CBV.
> If you only ever passed mutable objects around, you would think
> Python was call by reference.
True. But you'd still run into things that you can do with CBR in
other languages that you can't do in Python. That's not because Python
isn't CBR, it's because those other languages have things that Python
doens't have.
> And if you only ever passed immutable objects around, you would
> think Python was call by value.
You might. Then again, you might also understand the concepts well
enough to realize that there isn't any difference between CBR and CBV
when you're passing immutable objects.
> But since Python has both sorts of objects, it displays both sorts
> of behaviour.
Again, the mutability (or lack thereof) of the objects being passed
doesn't have anything to do with it. The things you can do in other
languages using CBR that you can't do in Python - like changing the
binding of a name in the callers namespace - you can't do whether the
name is bound to a mutable object or an immutable object. That Python
looks like CBV when you pass immutable objects is because CBR looks
like CBV when you pass immutable objects.
Bryan Olson
> Mike Meyer wrote:
[...]
>> Correct. What's stored in a list is a reference.
>
> Nonsense. What is stored in the list is an object.
According to the Python Language Reference:
Some objects contain references to other objects; these are
called containers. Examples of containers are tuples, lists
and dictionaries.
[http://docs.python.org/ref/objects.html]
--
--Bryan
Bryan Olson
> Steven D'Aprano writes:
[...]
>>Because the Original Poster said so! He said, to paraphrase, "Hey, I
>>thought Python was call by reference, but I tried this, and it didn't
>>work, what gives???"
>
>
> And he's right, and you're wrong. Look at the *code*. There isn't a
> single call in it. He may have said "call by reference"
Actually he didn't. Steven added "call by" in his paraphrase.
--
--Bryan
Donn Cave
| Mike Meyer wrote:
[... dragging out his own terminology hangup ]
|> Except this doesn't have *anything at all* to do with python being (or
|> not being) call by reference. This is a confusion about name binding
|> vs. assignment to a variable. The proper people to slap around for
|> this case are the ones who talk about assignment to a variable.
|
| Mike, you are wrong. Not about talk about "assigning to
| variables" being harmful, I agree with that. But this
| confusion is *exactly* about the call by reference
| misunderstanding. How do I know this?
|
| Because the Original Poster said so! He said, to
| paraphrase, "Hey, I thought Python was call by
| reference, but I tried this, and it didn't work, what
| gives???"
But he really did not SAY "call by reference"! The actual quote:
"I was under the assumption that everything in python was a refrence..."
and
"(After reading that 'everything' is a refrence.)"
Just give it up. And you too, Mike! "variable" is the commonplace
term for the things we fling around in our programs, and you have to
accept that we will use that word whether we're talking about Python,
Java, C or what have you. Of course the semantics of variables - how
they actually relate to values - may be radically different from one
language to another, but it universally plays essentially the same role
in programming (or mathematics), a thing whose value may vary according
to the logic of the program.
C and FORTRAN don't own this word, and it isn't just their version
against the way Python and some other languages do it. Each language
has its own angle on it, and they're all going to be called variables.
Donn Cave, do...@drizzle.com
Claudio Grondi
> On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
>
>
>>I was under the assumption that everything in python was a refrence...
>>
>>so if I code this:
>>lst = [1,2,3]
>>for i in lst:
>> if i==2:
>> i = 4
>>print lst
>>
>>I though the contents of lst would be modified.. (After reading that
>>'everything' is a refrence.)
>>...
>>Have I misunderstood something?
>
>
> It might help to do a translation to equivalent C:
>
> int _i1 = 1;
> int _i2 = 2;
> int _i3 = 3;
> int _i4 = 4;
> int* lst[NLST] = { &_i1,&_i2,&_i3 };
> int _idx; /* internal iterator */
> for (_idx = 0; _idx < NLST; ++_idx) {
> int *i = lst[_idx];
> if (*i == *_i2)
it is not.
Claudio
Ben Sizer
But, if you separate the calling mechanism from the assignment
mechanism, then Python does behave like every other call by reference
language. The problem is that people expect to then be able to change
the value of the referred object with the assignment operator. It's the
assignment semantics that differ from languages such as C++ and Java,
not the calling mechanism. In C++, assignment means copying a value. In
Python, assignment means reassigning a reference.
The unfortunate problem is that people think of 'pass by reference' in
association with 'assignment is a mutating operation', when really the
2 concepts are orthogonal and Python replaces the second with
'assignment is a reference reseating operation'. The only reason I
stress this is because with this in mind, Python is consistent, as
opposed to seeming to have 2 different modes of operation.
--
Ben Sizer
Dan Sommers
"Stuart D. Gathman" <stu...@bmsi.com> wrote:
> On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
>> I was under the assumption that everything in python was a refrence...
>>
>> so if I code this:
>> lst = [1,2,3]
>> for i in lst:
>> if i==2:
>> i = 4
>> print lst
>>
>> I though the contents of lst would be modified.. (After reading that
>> 'everything' is a refrence.)
>> ...
>> Have I misunderstood something?
> It might help to do a translation to equivalent C:
> int _i1 = 1;
> int _i2 = 2;
> int _i3 = 3;
> int _i4 = 4;
> int* lst[NLST] = { &_i1,&_i2,&_i3 };
Okay so far.
> int _idx; /* internal iterator */
> for (_idx = 0; _idx < NLST; ++_idx) {
> int *i = lst[_idx];
[snip]
That's the C idiom
for i in range(len(lst))
we all complain about here. How about (untested):
/*
iterate over the list, binding i to each item in succession;
_idx is internal to the interpreter;
separate the definition of i from the assignment of i for clarity
*/
int **_idx;
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i;
i = *_idx;
/* compare "the item to which i is bound" to "a constant" */
if( *i == *(&_i2) )
/* rebind i to _i4 */
i = &_i4;
}
> for (_idx = 0; _idx < NLST; ++_idx)
> printf("%d\n",*lst[_idx]);
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i = *_idx;
printf( "%d\n", *i );
}
Regards,
Dan
--
Dan Sommers
<http://www.tombstonezero.net/dan/>
David Murmann
> Stuart D. Gathman wrote:
>> for (_idx = 0; _idx < NLST; ++_idx) {
>> int *i = lst[_idx];
>> if (*i == *_i2)
> ^-- I have trouble with this line. Is it as is should be? I suppose
> it is not.
>
i think he meant
if (*i == _i2)
but i think python does
if (i == &_i2)
because there is only one integer object holding the value 2, so
it is sufficient to compare the addresses (i'm not sure about this,
perhaps someone smarter can clarify?).
--
David.
David Murmann
> but i think python does
>
> if (i == &_i2)
>
> because there is only one integer object holding the value 2, so
> it is sufficient to compare the addresses (i'm not sure about this,
> perhaps someone smarter can clarify?).
well, as far as i can see the relevant function is
static int
int_compare(PyIntObject *v, PyIntObject *w)
{
register long i = v->ob_ival;
register long j = w->ob_ival;
return (i < j) ? -1 : (i > j) ? 1 : 0;
}
in Objects/intobject.c, which does compare by value. so, this is
either special-cased elsewhere or not optimized (should/can it be?).
--
David.
Claudio Grondi
the moment my capacity to check it. Weren't it a good idea to use PyPy
to translate this Python code into C? It should exactly show how it
works, right?
I would be glad to hear how to accomplish this, because I have heard, it
is not easy to separate and use that part of PyPy which does the
translation.
Claudio
David Murmann
> well, as far as i can see the relevant function is
> either special-cased elsewhere or not optimized (should/can it be?).
it is special-cased, but still compares by value. the relevant
parts from "Python/ceval.c":
case COMPARE_OP:
if (PyInt_CheckExact(w) && PyInt_CheckExact(v)) {
a = PyInt_AS_LONG(v);
b = PyInt_AS_LONG(w);
switch (oparg) {
case PyCmp_EQ: res = a == b; break;
}
}
sorry if i am being noisy.
--
David.
Duncan Booth
> but i think python does
>
> if (i == &_i2)
>
> because there is only one integer object holding the value 2, so
> it is sufficient to compare the addresses (i'm not sure about this,
> perhaps someone smarter can clarify?).
No, Python itself only allocates one integer object holding the value 2,
but that isn't necessarily the only object which can compare equal to 2,
nor even the only integer object with the value of 2 (a C extension could
create another).
I seem to remember that at one time Python tried to 'optimise' the
comparison by saying that if the addresses are equal the objects are equal,
and if the addresses are not equal call the __eq__ method, but this doesn't
work in general as it is possible for an object to not be equal to itself.
(e.g. a NaN or a user defined class).
A suitable short equivalent for the comparison is:
if (*i == *(&_i2)) ...
but a closer equivalent of what Python really does might be:
object *_tmp = &_i2;
if ((PyInt_CheckExact(i) && PyInt_CheckExact(_tmp)) ?
*i==*_tmp : PyObject_RichCompare(i, _tmp, PyCmp_EQ)) ...
i.e. check the types (not that you could actually do that with the C
translation of the code) for two integers which fit in the range of C longs
and compare the values if possible, otherwise call some horrendously
complex comparison code.
David Murmann
> int **_idx;
> for( _idx = lst; _idx < lst + NLST; ++_idx ) {
> int *i;
> i = *_idx;
>
> /* compare "the item to which i is bound" to "a constant" */
> if( *i == *(&_i2) )
> /* rebind i to _i4 */
> i = &_i4;
> }
>
> int *i = *_idx;
> printf( "%d\n", *i );
> }
i don't see what is so fundamentally different in this version compared
to the one from Stuart Gathman (except it works, but i think it was a typo).
do you want to stress that python uses an iterator (if so, i would have
renamed _idx to _iter, because it ain't not index anymore)? whats the point
of not starting the for loop at 0? and is there a difference between
(*i == _idx) and (*i == *(&_idx))?
--
David.
Mike Meyer
> But, if you separate the calling mechanism from the assignment
> mechanism, then Python does behave like every other call by reference
> language. The problem is that people expect to then be able to change
> the value of the referred object with the assignment operator. It's the
> assignment semantics that differ from languages such as C++ and Java,
> not the calling mechanism. In C++, assignment means copying a value. In
> Python, assignment means reassigning a reference.
>
> The unfortunate problem is that people think of 'pass by reference' in
> association with 'assignment is a mutating operation', when really the
> 2 concepts are orthogonal and Python replaces the second with
> 'assignment is a reference reseating operation'. The only reason I
> stress this is because with this in mind, Python is consistent, as
> opposed to seeming to have 2 different modes of operation.
Exactly. The critical information is that python's assignment
statement is different from what people are used to from C/C++/etc.
While telling them that "You can't do call by reference because Python
is call by object" may be correct, it leaves out the critical
information. So they're liable to run into the same problem in a
different context - like the one the OP had.
Dan Sommers
David Murmann <david....@rwth-aachen.de> wrote:
> Dan Sommers schrieb:
>> int **_idx;
>> for( _idx = lst; _idx < lst + NLST; ++_idx ) {
>> int *i;
>> i = *_idx;
>> /* compare "the item to which i is bound" to "a constant" */
>> if( *i == *(&_i2) )
>> /* rebind i to _i4 */
>> i = &_i4;
>> }
>> for( _idx = lst; _idx < lst + NLST; ++_idx ) {
>> int *i = *_idx;
>> printf( "%d\n", *i );
>> }
For reference, Stuart Gathman's looks like this:
> > int _idx; /* internal iterator */
> > for (_idx = 0; _idx < NLST; ++_idx) {
> > int *i = lst[_idx];
> i don't see what is so fundamentally different in this version
> compared to the one from Stuart Gathman (except it works, but i think
> it was a typo) ...
I agree that Stuart's error was a typo.
> ... do you want to stress that python uses an iterator (if so, i would
> have renamed _idx to _iter, because it ain't not index anymore)? whats
> the point of not starting the for loop at 0? ...
I was trying to show iteration over a list of items, rather than indexed
access into a list of pointers, but it is no longer as clear to me as it
was at the time that I did a good job of that. ;-)
Another thing I wanted to show is that no amount of changing i would
have any effect on the list, and I missed it in Stuart's code (my
fault).
Or maybe my Python/C ratio is so high that seeing a loop variable used
as an index into a list or an array just bugs me. ;-)
_iter vs. _idx is a non-starter for me because of the baggage associated
with those words.
> ... and is there a difference between (*i == _idx) and (*i ==
> *(&_idx))?
Again, just the emphasis that the comparison is through bindings rather
than directly on the objects, to show that the bindings are completely
independent of the objects. I am pretty sure that there is no semantic
difference between *(&_idx) and _idx in the "normal" case, but there may
be some very odd corner cases if _idx were volatile and/or delcared
register.
Bengt Richter
>I was under the assumption that everything in python was a refrence...
>
>so if I code this:
>lst = [1,2,3]
>for i in lst:
> if i==2:
> i = 4
>print lst
>
>I though the contents of lst would be modified.. (After reading that
>'everything' is a refrence.)
>so it seems that in order to do this I need to code it like:
>
>lst = [1,2,3]
>for i in range(len(lst)):
> if lst[i] == 2:
> lst[i]=4
>print lst
>
>Have I misunderstood something?
>
Evidently, as others have already elaborated upon ;-)
I am wondering if it might be helpful to focus on expressions
first, and point out that the result is always a reference to
an object representing the value of the expression.
Since you always get a reference to a value-representing object,
assignment can only have meaning in terms of what you can do with that reference.
You can never assign a value per se. Therefore an assignment is
placing the reference in a place determined by the assignment target syntax.
The simplest is a bare name, and then we speak of binding a name to the
value-representing object. The name belongs to some context, and name space,
and an implementation will determine how to find the named binding site that
will hold the reference, depending on what name space is involved. E.g. the
binding site might be an optimized slot in a function's local name space.
Other assignment syntaxes also create bindings, and also identify what I am
calling binding sites. These sites may be within the representation of another object,
and located by some computation implementing the semantics, such as finding
an appropriate offset into the current representation of a list, or a hash-accessed
bucket/slot of a dict etc.
BTW, the implementation of finding a binding site for a non-bare-name binding
is typically a method call with the method and arguments determined by the trailing
end of the assignment target expression and the object whose method is to be found
determined by the expression up to the trailing end.
Binding site implementation is not part of the language, but the semantics of
binding objects 1:1 or 1:many is.
Mutable objects have assignable binding sites. Immutable objects do not (except
for the "assignment" of initial binding references during their creation). Containing objects
contain binding sites, not values. Binding sites must have references to some
value-representing object or other if they semantically exist, though of course an implementation
may manage space any way convenient if semantics are preserved.
Going back to an expression again, when e.g., a name is evaluated, the name identifies
a binding site somwhere, and the value of any expression (including bare name) is, as we said at the start,
"a reference to an object representing the value of the expression."
In the case of a bare name, we have that reference as soon as we have identified the binding site
associated with the name, since that contains a reference to a value-representing object. If this reference
is now assigned to a binding site identified by another bare name, we have two bare names with respective
associated binding sites having references referring to the same value-representing object.
IOW, python shares value representations by default. Including mutable value representations. Which
means if you want a distinct mutable value equal to a given one, you need an expression that does
more than evaluate to find the reference to the given one -- i.e., creates a new value-representing object
using the given for information, and returns a reference the new.
To a C programmer, it will be tempting to say that "binding sites" are just pointer-data locations,
and references are pointers, and "value-representing-object" is just malloc'ed space for some custom
struct or array or both. This could be true for a particular implementation, but I think "binding site"
and "reference" as I have just used them, are also abstract concepts, capable of alternative, non-C
implementation of the python semantics. So I thought I'd try this as a possible terminology, to see if
it makes it any easier for anyone ;-)
Regards,
Bengt Richter
Steven D'Aprano
Is it so hard to understand that the word "reference" has a general,
imprecise meaning in common English (which is clearly how the quote
above is using the word) while still having in the context of assignment
and argument passing a more precise meaning which is dangerously
misleading?
Words are important -- not only for what they mean, but for what the
connotations they carry. For people who come to Python from C-like
languages, the word "reference" means something that is just not true in
the context of Python's behaviour. That's why people come to Python with a
frame that tells that what call by reference implies ("I can do this...")
and then they discover that they often *can't* do that.
It is a crying shame, because reference is a nice, generic word that just
cries out to be used in a nice, generic way, as the Python doc you quoted
is doing. But unfortunately, the term "reference" has been co-opted by C
programmers to effectively mean "pointer to a value".
The aim of the Python doc authors is to communicate information about
Python effectively. That means they have to be aware of words'
connotations and the mental frames they evoke. In the same way a good
programmer must work around bugs in the operating system or libraries, a
good author must work around bugs in people's mental frames which will
cause misunderstanding. That's why "reference" is a bad word to use in the
context of Python containers and argument handling: far from communicating
correct information effectively, it gives many readers a misleading
assumption about how Python code will behave.
The proof of this is the number of times people write to this newsgroup
confused about "call by reference" -- not because they came to Python with
assumptions about its behaviour, but because somebody told that Python was
"call by reference". Or that lists contain "references" to other objects.
In English, I can say "I'm writing a book on Russian history. I've
included a reference to Smith's classic work on the tsars of Russia."
That's a good, generic use of the word "reference". Nobody thinks that
anything I do to modify my book will effect Smith's work. Books don't work
that way.
But in programming, things do work that way. If my class Book contains a
reference to Smith's classic work, I can modify it. (Unless the language
deliberately restricts my ability to modify certain objects, as Python
does with immutable objects.)
That's what programmers expect when you talk about references, especially
if they come from a C (or Pascal) background. In Python, sometimes that's
true, and sometimes it is not, and the only way to tell is by looking at
the object itself, not by thinking about Python's high-level behaviour.
Thinking about Python's behaviour ("it always passes references to
objects") will invoke misleading frames in many programmers' minds. The
word "reference" is misleading and should be avoided, because what the
average non-Python programmer understands by the word is different from
what the experienced Pythonista understands by it.
If we were writing academic papers, we could define "call by reference"
and "objects contain references" any way we liked, and it would be the
responsibility of the readers to ensure they understood *our* meaning. But
we're not -- we're communicating information to ordinary programmers, many
of whom are kids still in school, not academics. Many of them will be
coming to Python with preconceived ideas of the meaning of "call by
reference", "assign a variable", etc. It is *our* responsibility to use
language that will not be misleading, that will not invoke incorrect
frames, that will not lead them up the garden path and be surprised by
Python's behaviour.
If we say "Python is call be reference" (or call by value, as many people
also say) we *know* the consequence will be newbies writing in saying "I
was told Python is call by reference, so I did this, and it didn't work,
is that a bug in Python? What is wrong?" It is not a bug in Python, it is
a bug in their mental model of how Python works, and we put that bug in
their head. Every time that happens, it is *our* fault, not theirs, for
using language guaranteed to mislead. If we use an unfamiliar term like
"call by object", the reader has no preconceived understanding of what it
means and won't be lead to incorrect assumptions.
We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?
--
Steven.
Steven D'Aprano
>> And if you only ever passed immutable objects around, you would
>> think Python was call by value.
>
> You might. Then again, you might also understand the concepts well
> enough to realize that there isn't any difference between CBR and CBV
> when you're passing immutable objects.
Consider this:
def do_nothing(x):
pass
huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)
If Python made a copy of huge_tuple before passing it to the function, you
would notice.
--
Steven.
Donn Cave
Steven D'Aprano <st...@REMOVETHIScyber.com.au> wrote:
...
> We know that this will happen because it has happened time and time again
> in the past. Are we incapable of learning from experience? Are we
> intelligent sentient beings or do we just parrot what was said in the
> past with no concern for the consequences of what we say?
The latter. I know this from the number of times I have read
that parameter passing has something to do with whether the
parameter is a mutable or immutable object.
Donn Cave, do...@u.washington.edu
Steven D'Aprano
> While telling them that "You can't do call by reference because Python
> is call by object" may be correct,
Good to see you finally concede it.
> it leaves out the critical information.
As does "call by reference" or "call by value". No three word phrase can
possibly explain Python's behaviour, or any other language for that
matter. You have to know what "call by whatever" *implies* for it to be
meaningful.
> So they're liable to run into the same problem in a
> different context - like the one the OP had.
That doesn't follow.
The problem isn't that talk about references is *wrong*, but that it is
misleading. Where "reference" is being used just as a generic term, it is
barely acceptable, but any time it is used in a context where it could be
misconstrued is dangerous precisely because people WILL misunderstand it,
because they *think* they know what references (assignment, call by
reference, whatever) means in this context.
The fact that call by object is unfamiliar is precisely its advantage.
Instead of people jumping to conclusions about what it means, like they do
with "reference", they will stop and think and if need be look for further
information about Python's object model. At the very least, they won't say
"Everything in Python is a reference, so this should work, but it
doesn't".
--
Steven.
Mike Meyer
> objects") will invoke misleading frames in many programmers' minds. The
> word "reference" is misleading and should be avoided, because what the
> average non-Python programmer understands by the word is different from
> what the experienced Pythonista understands by it.
Yes, but it's not misleading because "reference" doesn't mean what
they think it means. "reference" means the same thing in Python as it
does in C and similar languages. It's misleading because "variables"
and "assignment" in Python don't do the things they do in C. Some of
the places that this shows up is when you're dealing with call by
reference, or with references in a list.
> We know that this will happen because it has happened time and time again
> in the past. Are we incapable of learning from experience? Are we
> intelligent sentient beings or do we just parrot what was said in the
> past with no concern for the consequences of what we say?
No, we learned to say "binding" and "names". We also learned that
while giving newbies false information like "Python doesn't have
references" or "Python isn't call-by-reference" will help them get
past the immediate problem, it sets them up to be confused in some
other area where variables and assignments don't do what they
think. So we've learned to deal with the disease instead of the
symptoms, and tell them how variables in Python behave differently
from variables in C, or Pascal, or ALGOL, or FORTRAN, or whatever.
Mike Meyer
>> While telling them that "You can't do call by reference because Python
>> is call by object" may be correct,
> Good to see you finally concede it.
I'm not conceeding anything, because I never said otherwise. In
particular, I still claim that Pythons behavior when passing objects
as parameters is identical to call-by-reference for all the objects
that you can generate in Python.
>> So they're liable to run into the same problem in a
>> different context - like the one the OP had.
> That doesn't follow.
You're right. That observation is based on watching people repeatedly
beat their head against the difference between variable assignment in
other languages and name binding in object-based languages. Until you
get that difference into their heads, they're going to have
problems. What you use to describe the parameter passing mechanism is
immaterial to the real problem.
> The problem isn't that talk about references is *wrong*, but that it is
> misleading. Where "reference" is being used just as a generic term, it is
> barely acceptable, but any time it is used in a context where it could be
> misconstrued is dangerous precisely because people WILL misunderstand it,
> because they *think* they know what references (assignment, call by
> reference, whatever) means in this context.
This is where we disagree. I think their understanding of references
is dead on. What's broken is their understanding of what variables are
and what assignments mean. Once you fix that, the rest falls into
place.
> The fact that call by object is unfamiliar is precisely its advantage.
> Instead of people jumping to conclusions about what it means, like they do
> with "reference", they will stop and think and if need be look for further
> information about Python's object model. At the very least, they won't say
> "Everything in Python is a reference, so this should work, but it
> doesn't".
What you're advocating is intentionally misleading people to deal with
a symptom. I'd rather give them enough truth to deal with the disease.
Terry Hancock
"Steven D'Aprano" <st...@REMOVETHIScyber.com.au> wrote:
Which succinctly demonstrates precisely why a newbie
*should* be told that Python passes references instead of
values. Without this basic knowledge, the newcomer can't
possibly be expected to make intelligent choices of
algorithm.
But it should probably also be made clear that
"reference" means a label assigned to an object, and not a
variable containing a memory location (which is what a C
"pointer" is).
The reference is usually a simple name, but it can also be
an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because they
are immutable, and so don't have assignable references.
Cheers,
Terry
--
Terry Hancock (han...@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com
Ernst Noch
> Steven D'Aprano <st...@REMOVETHIScyber.com.au> writes:
>
>>Thinking about Python's behaviour ("it always passes references to
>>objects") will invoke misleading frames in many programmers' minds. The
>>word "reference" is misleading and should be avoided, because what the
>>average non-Python programmer understands by the word is different from
>>what the experienced Pythonista understands by it.
>
>
> Yes, but it's not misleading because "reference" doesn't mean what
> they think it means. "reference" means the same thing in Python as it
> does in C and similar languages. It's misleading because "variables"
> and "assignment" in Python don't do the things they do in C. Some of
> the places that this shows up is when you're dealing with call by
> reference, or with references in a list.
Maybe next time showing something like the following trivial snippet
might help demonstrate that the core of the matter doesn't is not the
way python treats parameters?
>>> def func(param, what):
if what:
param['foo'] = 'changed'
else:
temp = param['foo'] # temp is _not_ a reference!
temp = 'changed'
Steven D'Aprano
>> Consider this:
>>
>> def do_nothing(x):
>> pass
>>
>> huge_tuple = (None,) * 10000**4
>> do_nothing(huge_tuple)
>>
>> If Python made a copy of huge_tuple before passing it to
>> the function, you would notice.
>
> Which succinctly demonstrates precisely why a newbie
> *should* be told that Python passes references instead of
> values.
"I don't like Chinese food, and pizza gives me gas. Let's have a burger,"
says I.
And Terry helpfully answers: "Pizza gives you gas? Well there you go then,
that's precisely why we have to have Chinese food."
I have never suggested that we should tell newbies that Python is call
by value, and I deny that the only two possible choices for usefully
describing Python's behaviour is by CBV and CBR. The fact that Terry could
read this thread and still imagine that there is such a dichotomy is
worrying.
> But it should probably also be made clear that
> "reference" means a label assigned to an object, and not a
> variable containing a memory location (which is what a C
> "pointer" is).
>
> The reference is usually a simple name, but it can also be
> an container expression following list, dictionary,
> or class instance (spam, spam[0], spam['eggs'], or
> spam.ham, for example). Not tuple or string because they
> are immutable, and so don't have assignable references.
Are you seriously suggesting that you can't include tuples in a list, or
pass them to functions? Because that's what it sounds like you're saying:
"Python passes references. The reference is usually a simple name, but it
can also be a container... Not tuple or string because they are immutable."
If you don't mean to say that Python can't pass strings to functions, what
do you mean to say?
I'll tell you what I say: Python passes objects to functions or
assignments.
Does this mean that the object is copied? No, I didn't say it copies
objects. I left the nature of the passing mechanism unspoken, which is how
it should be because it is an implementation detail.
Does this mean that the object can be modified in place? Certainly not,
since that depends on the object, not on the nature of Python's high-level
behaviour. It emphasises the object oriented nature of Python, and by its
very language warns the reader not to assume that Python behaves
identically to other languages. It invites further questions, rather than
lulling the reader into jumping to false conclusions.
The emphasis is on the *object*, not the passing mechanism. In C,
everything is mutable, and whether you can change an item depends on
whether you are working with a reference to that item or a copy of the
item. That's a language issue, not a data issue -- in some languages,
you can even choose whether to pass a reference to a function or a copy
of the value.
In Python, whether or not you can change an object depends on the object,
not the language itself: it is a data issue.
--
Steven.
bon...@gmail.com
Steven D'Aprano wrote:
> I'll tell you what I say: Python passes objects to functions or
> assignments.
>
maintain by the system, identified by id().
Steven D'Aprano
And that C sense is precisely why people try using C idioms in Python and
then are surprised that they don't behave as they would in C.
So how about we all agree to stop implying that Python behaves like C, and
try to teach people that Python behaves like Python?
--
Steven.
bon...@gmail.com
close as possible to people speaking that language.
"passed by object" can just as easilly be interpreted as "the function
received a seperate copy", with the caller's copy untouched.
bon...@gmail.com
Steven D'Aprano wrote:
> But in programming, things do work that way. If my class Book contains a
> reference to Smith's classic work, I can modify it. (Unless the language
> deliberately restricts my ability to modify certain objects, as Python
> does with immutable objects.)
>
> That's what programmers expect when you talk about references, especially
> if they come from a C (or Pascal) background. In Python, sometimes that's
> true, and sometimes it is not, and the only way to tell is by looking at
> the object itself, not by thinking about Python's high-level behaviour.
I believe one can restrict modification to pointer/reference parameter
passing to C function, so C programmer shouldn't have problem knowing
that reference doesn't not necessary mean you can modify it.
Mike Meyer
> Maybe next time showing something like the following trivial snippet
> might help demonstrate that the core of the matter doesn't is not the
> way python treats parameters?
Did you insert an extra "doesn't" in that? If so, then I agree about
what isn't the core of the matter.
On the other hand, the snippet doesn't help without an
explanation. Especially with misleading comments.
>>>> def func(param, what):
> if what:
> param['foo'] = 'changed'
> else:
> temp = param['foo'] # temp is _not_ a reference!
Except temp *is* a reference. What it's not is a reference to
param['foo']. Instead, it's a reference to the same object that
param['foo'] is a reference to.
> temp = 'changed'
Ernst Noch
> Ernst Noch <en...@gmx.net> writes:
>
>>Maybe next time showing something like the following trivial snippet
>>might help demonstrate that the core of the matter doesn't is not the
>>way python treats parameters?
>
>
> Did you insert an extra "doesn't" in that? If so, then I agree about
> what isn't the core of the matter.
Uhm, yes. Seems the whole message was written too hastily.
>
> On the other hand, the snippet doesn't help without an
> explanation. Especially with misleading comments.
Absolutely, it was mainly meant as an example to demonstrate in practice
that the way python passes parameters has nothing to do with what happens.
>
>
>>>>>def func(param, what):
>>
>> if what:
>> param['foo'] = 'changed'
>> else:
>> temp = param['foo'] # temp is _not_ a reference!
>
>
> Except temp *is* a reference. What it's not is a reference to
> param['foo']. Instead, it's a reference to the same object that
> param['foo'] is a reference to.
>
Oh dear, yes. I inserted that comment as an afterthought.
Donn Cave
Steven D'Aprano <st...@REMOVETHIScyber.com.au> wrote:
> I'll tell you what I say: Python passes objects to functions or
> assignments.
>
> Does this mean that the object is copied? No, I didn't say it copies
> objects. I left the nature of the passing mechanism unspoken, which is how
> it should be because it is an implementation detail.
Thus conveniently deferring the necessary explanation of how
it actually works, which will not be too hard but resists
simplistic "use this word" solutions.
> The emphasis is on the *object*, not the passing mechanism. In C,
> everything is mutable, and whether you can change an item depends on
> whether you are working with a reference to that item or a copy of the
> item. That's a language issue, not a data issue -- in some languages,
> you can even choose whether to pass a reference to a function or a copy
> of the value.
>
> In Python, whether or not you can change an object depends on the object,
> not the language itself: it is a data issue.
Oh, baloney. Same in C -
$ cc -c /tmp/t.c
/tmp/t.c: In function 'churk':
/tmp/t.c:4: error: assignment of read-only location
As long as you pass by pointer, it is much the same -
void
churk(const char *data)
{
data[0] = 'C'; /* error - modify const object */
data = "C"; /* no problem here - rebind pointer parameter */
}
Maybe the cure for hardened C programmers who aren't
getting it is to emphasize the pointer angle - and note
that there isn't any way to write "*i = 4". "Everything
is a pointer", let's say -- or maybe, "Everything is a
reference" would be smarter.
C programmers are used to seeing their data called
objects, too. (Not to mention the unlucky many of
us who learned OOP via C++.)
Donn Cave, do...@u.washington.edu
Terry Hancock
> >> Consider this:
> >>
> >> def do_nothing(x):
> >> pass
> >>
> >> huge_tuple = (None,) * 10000**4
> >> do_nothing(huge_tuple)
> >>
> >> If Python made a copy of huge_tuple before passing it
> >to > the function, you would notice.
> >
> > Which succinctly demonstrates precisely why a newbie
> > *should* be told that Python passes references instead
> > of values.
>
> "I don't like Chinese food, and pizza gives me gas. Let's
> have a burger," says I.
>
> And Terry helpfully answers: "Pizza gives you gas? Well
> there you go then, that's precisely why we have to have
> Chinese food."
An annoyingly banal, off-topic, and incorrect analogy.
> I have never suggested that we should tell newbies that
> Python is call by value, and I deny that the only two
> possible choices for usefully describing Python's
> behaviour is by CBV and CBR. The fact that Terry could
> read this thread and still imagine that there is such a
> dichotomy is worrying.
I didn't say you did, and it's very worrying that you could
read my post and not understand that. You want to get
tetchy? I can *be* tetchy, if that's what you want. ;-)
As far as I am concerned "call by object" which is what you
*did* propose implies "call by value" to anyone who is just
learning the language, and that is the audience we're
talking about.
I mean, *I* know what you mean when you say "Python is 'call
by object'" just as *you* know what I mean when I say
"Python is 'call by reference'", but where does that leave
Mr. Newbie?
Sorry, but the burgers give him gas, too.
> > But it should probably also be made clear that
> > "reference" means a label assigned to an object, and not
> > a variable containing a memory location (which is what a
> > C "pointer" is).
> >
> > The reference is usually a simple name, but it can also
> > be an container expression following list, dictionary,
> > or class instance (spam, spam[0], spam['eggs'], or
> > spam.ham, for example). Not tuple or string because
> > they are immutable, and so don't have assignable
> > references.
>
> Are you seriously suggesting that you can't include tuples
> in a list, or pass them to functions? Because that's what
> it sounds like you're saying: "Python passes references.
> The reference is usually a simple name, but it can also be
> a container... Not tuple or string because they are
> immutable."
>
> If you don't mean to say that Python can't pass strings to
> functions, what do you mean to say?
No, don't be intentionally thick.
I mean that you cannot dereference a string or tuple in
order to assign elements of them -- which is another way of
saying they are immutable.
> I'll tell you what I say: Python passes objects to
> functions or assignments.
>
> Does this mean that the object is copied? No, I didn't say
> it copies objects. I left the nature of the passing
> mechanism unspoken, which is how it should be because it
> is an implementation detail.
And that is where we part ways, my friend. Because
"implementation details" do matter when they make a
difference between execution in 10 seconds and executions in
10 days. Even a newbie is going to be hesitant to do things
that look like they are going to do the latter. He still
needs to have a reasonably correct mental model of what his
program is going to do.
> Does this mean that the object can be modified in place?
> Certainly not, since that depends on the object, not on
> the nature of Python's high-level behaviour. It emphasises
> the object oriented nature of Python, and by its very
> language warns the reader not to assume that Python
> behaves identically to other languages. It invites further
> questions, rather than lulling the reader into jumping to
> false conclusions.
Chigaimasu! It just leads to *different* false conclusions.
> The emphasis is on the *object*, not the passing
> mechanism.[...]
> In Python, whether or not you can change an object depends
> on the object, not the language itself: it is a data
> issue.
True, but the passing mechanism still makes a difference in
understanding how Python programs operate.
Of course, this conversation is starting to get extremely
silly.
Mike Meyer
> As far as I am concerned "call by object" which is what you
> *did* propose implies "call by value" to anyone who is just
> learning the language, and that is the audience we're
> talking about.
From what I can tell, Liskov proposed *three* different names for
passing references to objects: call-by-sharing, call-by-object, and
call-by-object-reference.
Call by object is the worst choice among the three, because "object"
has such a vague meaning, so you never know what implications someone
will come away with. Of the other two, I like call by object
reference, because it's clear you're passing a reference to an object,
and it doesn't really matter what an object is in this case.
Bryan Olson
> On Thu, 05 Jan 2006 05:21:24 +0000, Bryan Olson wrote:
>
>
>>Steven D'Aprano wrote:
>>
>>>Mike Meyer wrote:
>>
>>[...]
>>
>>>>Correct. What's stored in a list is a reference.
>>>
>>>Nonsense. What is stored in the list is an object.
>>
>>According to the Python Language Reference:
>>
>> Some objects contain references to other objects; these are
>> called containers. Examples of containers are tuples, lists
>> and dictionaries.
>> [http://docs.python.org/ref/objects.html]
>
>
> Is it so hard to understand that the word "reference" has a general,
> imprecise meaning in common English (which is clearly how the quote
> above is using the word) while still having in the context of assignment
> and argument passing a more precise meaning which is dangerously
> misleading?
What's pretty easy to understand at this point, is that Mike
was right and you were wrong. "Reference" has a precise meaning
here, it's what the Python language uses, and contrary to your
reporting, it's what the rest of the discipline uses.
> Words are important -- not only for what they mean, but for what the
> connotations they carry. For people who come to Python from C-like
> languages, the word "reference" means something that is just not true in
> the context of Python's behaviour.
Wrong. C does not have references, and the Python use is consistent
with the rest of computer science. You seem to have read in things
that it does not mean. Fix *your* thinking.
That's why people come to Python with a
> frame that tells that what call by reference implies ("I can do this...")
> and then they discover that they often *can't* do that.
I'm sorry if you got confused, but please don't project it on
the rest of the discipline. C does not have even references.
[...]
> Thinking about Python's behaviour ("it always passes references to
> objects")
Just fix your thinking and don't attribute these problems to
others.
[...]
> If we were writing academic papers, we could define "call by reference"
> and "objects contain references" any way we liked,
That would be a terrible thing to do. Just learn to use the
meaning accepted in the discipline, and used in the Python doc.
--
--Bryan
Steven D'Aprano
> Call by object is the worst choice among the three, because "object"
> has such a vague meaning, so you never know what implications someone
> will come away with.
So very unlike "call by reference", right?
--
Steven.
Steven D'Aprano
> Wrong. C does not have references, and the Python use is consistent
> with the rest of computer science. You seem to have read in things
> that it does not mean. Fix *your* thinking.
Bryan, I'll admit that I'm no C/C++ programmer, and I frequently assume
that if X is true for C++ it is also true for C, and vice versa. I
appreciate being corrected when I make a mistake.
However, in this case, I think you are confused by the term references. Of
course pure C does not have the C++ data structure known as "references",
but that is hardly the only meaning of the word. "Reference" is a generic
term for something which refers to something else, not just in plain
English but also in computer science. Pointers are references. Indexes
into a table are references. URLs are references. File names and file
handles are references, as are keys from databases, and even raw memory
addresses.
http://en.wikipedia.org/wiki/Reference_(computer_science)
I'll admit that my position on this question has changed. I no longer say
that "everything in Python is a reference" is *false*, not in the weak,
generic sense. How can it be? In the generic sense, Python names are
references. (Under the hood, names may even be implemented as C++
references for all I know.) I will still argue that the term reference is
harmful, not because it is technically incorrect, but because it activates
incorrect mental models of Python's high-level behaviour in too many
people.
>> That's why people come to Python with a
>> frame that tells that what call by reference implies ("I can do this...")
>> and then they discover that they often *can't* do that.
>
> I'm sorry if you got confused, but please don't project it on
> the rest of the discipline.
Perhaps you should check out the beginning of the thread before making
any additional comments. It wasn't me who was confused and asked "Hey
what's going on? I was told everything in Python is a reference, but when
I do this it doesn't work." I'm not projecting anything on anyone.
--
Steven.
Bryan Olson
> Bryan Olson wrote:
>>Wrong. C does not have references, and the Python use is consistent
>>with the rest of computer science. You seem to have read in things
>>that it does not mean. Fix *your* thinking.
>
>
> Bryan, I'll admit that I'm no C/C++ programmer, and I frequently assume
> that if X is true for C++ it is also true for C, and vice versa. I
> appreciate being corrected when I make a mistake.
>
> However, in this case, I think you are confused by the term references.
No, though I should have had "references" in quotes; C doesn't
call anything a reference, so saying it confuses people about
what references are doesn't make sense.
[...]
> I'll admit that my position on this question has changed. I no longer say
> that "everything in Python is a reference" is *false*, not in the weak,
> generic sense.
Of course it's false. Is indentation a reference? Is a comment
a reference? The bad term there is "everything", not "reference".
> How can it be? In the generic sense, Python names are
> references. (Under the hood, names may even be implemented as C++
> references for all I know.) I will still argue that the term reference is
> harmful, not because it is technically incorrect, but because it activates
> incorrect mental models of Python's high-level behaviour in too many
> people.
Mike wrote that lists contain references. You said that was
nonsense as they contain objects. His description was right and
consistent with Python behavior. Yours was wrong and inconsistent
with Python behavior. List storing references is the *right*
mental model.
>>>That's why people come to Python with a
>>>frame that tells that what call by reference implies ("I can do this...")
>>>and then they discover that they often *can't* do that.
>>
>>I'm sorry if you got confused, but please don't project it on
>>the rest of the discipline.
>
> Perhaps you should check out the beginning of the thread before making
> any additional comments. It wasn't me who was confused and asked "Hey
> what's going on? I was told everything in Python is a reference, but when
> I do this it doesn't work." I'm not projecting anything on anyone.
Read more carefully. His problem was in understanding variable
assignment in C versus Python: storage in a location versus name
binding. He got some good explanations. Unfortunately there was
also quite a bit of wrong reporting, which some of us are trying
to correct.
--
--Bryan
Alex Martelli
...
> assignment semantics that differ from languages such as C++ and Java,
> not the calling mechanism. In C++, assignment means copying a value. In
> Python, assignment means reassigning a reference.
And in Java, it means just the same as in Python (with some unfortunate
exceptions, in Java, for elementary types such as int -- but for all
normal types, the meaning of assignment and parameter passing is just
the same in Java as in Python).
Considering that Java may have become the language most used for first
courses in programming, it's unfortunate to propagate disinformation
about its assignment semantics differing from Python's -- it will
confuse people who know Java, and there are many of those.
Alex
Fredrik Lundh
> > Words are important -- not only for what they mean, but for what the
> > connotations they carry. For people who come to Python from C-like
> > languages, the word "reference" means something that is just not true in
> > the context of Python's behaviour.
>
> Wrong. C does not have references
Unless they've changed it since the drafts, the ANSI C specification uses
the word "reference" to describe how pointers work:
A pointer type describes an object
whose value provides a reference to an entity of the
referenced type. A pointer type derived from the
referenced type T is sometimes called ``pointer to T''.
The construction of a pointer type from a referenced
type is called ``pointer type derivation''.
so unless you're using some odd dialect of C that doesn't support pointers,
your C implementation sure has references.
> and the Python use is consistent with the rest of computer science.
The problem isn't the word "reference" in itself, the problem is when people
are implying that "since Python passes object references to functions, it's
using call by reference".
In ordinary CS, "call by reference" generally means that the function is
handed a reference to the *variable* holding the *value*. You can do
this explicitly in C (by passing in &arg instead of arg), and you can do
this in C++, but there's no way to do this in Python -- Python variables
hold objects, not values.
Saying that Python uses "call by value" is probably more correct (where
the values are references), but that's even more confusing. (see endless
earlier threads for more on this).
> That would be a terrible thing to do. Just learn to use the
> meaning accepted in the discipline, and used in the Python doc.
afaik, the Python Language Reference never defines the word "reference".
It carefully defines words like "object" and "value", though, and terms like
"call by object" or "call by object reference" are perfectly understandable
if you use the words as they are defined in the language reference.
</F>
Terry Hancock
Mike Meyer <m...@mired.org> wrote:
> From what I can tell, Liskov proposed *three* different
> names for
> passing references to objects: call-by-sharing,
> call-by-object, and call-by-object-reference.
"Call by object reference" makes the most sense to me. Names
in Python are object references: they refer to objects. You
might almost say "call by name" but that sort of implies
that you are passing the strings around (which is generally
untrue), and it doesn't convey what happens when the name is
something complex like "ham['spam'][0].eggs()[42]" (i.e.
"name" carries the conotation of being a simple string,
although you could argue that it doesn't have to mean that).
Mike Meyer
Right. All references are objects, so "call by object" includes all
the possibilities of "call by reference" as a subset. Not all objects
are references, so "call by reference", so it's a proper subset.
Mike Meyer
> In ordinary CS, "call by reference" generally means that the function is
> handed a reference to the *variable* holding the *value*.
That's the strictest definition of "call-by-reference". It's got a
major problem in that it means doing (with C syntax) "foo(&l[23])"
isn't CBR, because I didn't pass a reference to a *variable*. For
Python to be CBR, you have to use a broad definition of CBR
> Saying that Python uses "call by value" is probably more correct (where
> the values are references), but that's even more confusing.
I wouldn't say 'more correct', because of the above. But since *all*
paramater passing mechanisms pass a value of some kind - even
call-by-substitution - they're all call by value with that
definition. That makes the phrase pretty much useless.
The problem is that CBR and CBV were defined for langauges with
variables that had addresses, and assignments that copied values into
those addresses. The definitions were stretched by various groups to
fit their language, and some of the definitions disagree about edge
cases. The calling mechanisms of languages that have variables that
are bound to objects are all edge cases.
Mike Meyer
> On Sat, 07 Jan 2006 01:29:46 -0500
> Mike Meyer <m...@mired.org> wrote:
>> From what I can tell, Liskov proposed *three* different
>> names for
>> passing references to objects: call-by-sharing,
>> call-by-object, and call-by-object-reference.
> "Call by object reference" makes the most sense to me. Names
> in Python are object references: they refer to objects. You
> might almost say "call by name" but that sort of implies
> that you are passing the strings around (which is generally
> untrue)
"Call by name" has a specific meaning that is different from passing
strings around. CBName (and CBNeed) is generally implemented by
passing thunks. In Python, you could do CBN by passing tuples of
(string, globals(), locals()) around - which probably makes enough
noise when it hits the stack to qualify as a thunk.
Aahz
context]
In article <86bqyn7...@bhuda.mired.org>, Mike Meyer <m...@mired.org> wrote:
>
>Right. All references are objects, so "call by object" includes all
>the possibilities of "call by reference" as a subset. Not all objects
>are references, so "call by reference", so it's a proper subset.
Wrong. All references are *to* objects, maybe. There are no objects
that "are" references -- objects may *contain* references. You're
losing sight of the fact that objects and references are really two
completely different things in Python.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"19. A language that doesn't affect the way you think about programming,
is not worth knowing." --Alan Perlis
Anders Hammarquist
Fredrik Lundh <fre...@pythonware.com> wrote:
>> and the Python use is consistent with the rest of computer science.
>
>The problem isn't the word "reference" in itself, the problem is when people
>are implying that "since Python passes object references to functions, it's
>using call by reference".
Actually, I think the problem is not in the "call" bit. Python behaves
more or less like all other call by reference languages in the call. What
is confusing people is what happens when you *assign* to a function argument.
Those confused are expecting Python function arguments to behave just
like C++ references (which are just syntactic sugar for a dereferenced
pointer) or an Ada "in out" parameter when assigned. Python doesn't,
because in Python assignment assigns a new reference to the name,
rather than changing the referenced object.
/Anders
--
-- Of course I'm crazy, but that doesn't mean I'm wrong.
Anders Hammarquist | i...@cd.chalmers.se
Physics student, Chalmers University of Technology, | Hem: +46 31 88 48 50
G|teborg, Sweden. RADIO: SM6XMM and N2JGL | Mob: +46 707 27 86 87
ru...@yahoo.com
Fredrik Lundh wrote:
...snip...
> afaik, the Python Language Reference never defines the word "reference".
> It carefully defines words like "object" and "value", though, and terms like
> "call by object" or "call by object reference" are perfectly understandable
> if you use the words as they are defined in the language reference.
It (sec. 3.1, "Objects, values and types") is not what I would
call a good definition . About values it says only
- that they are something that all objects have.
- they can be mutable or immutable.
It then has a few sentences about mutability, so after reading
it you will know that, whatever a value is, it can be changed
for some objects but not others. But what exactly it is that is
being changed is still a mystery.
Further down it talks about container objects "containing"
references to other objects and of those being part of it's
value. Part? What does it mean to contain? Can one
determine from reading this, if an attribute value is part
of the object's value? (I couldn't).
On my list on Python Doc problems that need fixing, is
"defintion of object values" and it has been on there
for nearly a year.
So I don't think referring people to the Language Reference
is a good way to help them understand Python. Python
badly needs a rewritten Language Reference.
Donn Cave
| Fredrik Lundh wrote:
| ...snip...
| > afaik, the Python Language Reference never defines the word "reference".
| > It carefully defines words like "object" and "value", though, and terms like
| > "call by object" or "call by object reference" are perfectly understandable
| > if you use the words as they are defined in the language reference.
|
| It (sec. 3.1, "Objects, values and types") is not what I would
| call a good definition . About values it says only
|
| - that they are something that all objects have.
| - they can be mutable or immutable.
|
| It then has a few sentences about mutability, so after reading
| it you will know that, whatever a value is, it can be changed
| for some objects but not others. But what exactly it is that is
| being changed is still a mystery.
| Further down it talks about container objects "containing"
| references to other objects and of those being part of it's
| value. Part? What does it mean to contain? Can one
| determine from reading this, if an attribute value is part
| of the object's value? (I couldn't).
|
| On my list on Python Doc problems that need fixing, is
| "defintion of object values" and it has been on there
| for nearly a year.
So you've had time to think about how you would define value, in a
few words. Any ideas?
I find the topic difficult, myself. I think you really have to apply
some context to the question, so there may not be any satisfactory
definition for the language reference.
But maybe it would be simple with the right focus. If we could somehow
define value, how would that help? I mean, presumably we need to
understand all this stuff because we want to write some software, and
if we dive in without understanding, our attempts will be plagued with
conceptual errors. Is there something about value in particular that
seems to be a problem here? ``No, you idiot, that's not a value -
THIS is a value!''
Donn Cave, do...@drizzle.com
Patrick Maupin
> This is where we disagree. I think their understanding of references
> is dead on. What's broken is their understanding of what variables are
> and what assignments mean. Once you fix that, the rest falls into
> place.
>
> (Steven D'Aprano wrote:)
> > The fact that call by object is unfamiliar is precisely its advantage.
> > Instead of people jumping to conclusions about what it means, like they do
> > with "reference", they will stop and think and if need be look for further
> > information about Python's object model. At the very least, they won't say
> > "Everything in Python is a reference, so this should work, but it
> > doesn't".
>
> What you're advocating is intentionally misleading people to deal with
> a symptom. I'd rather give them enough truth to deal with the disease.
I agree completely. I have personally "converted" around a half-dozen
or so coworkers to Python over the course of the last three or four
years. The conversion takes place in stages. Usually, I have written
a small script for them which they later want to enhance, so they come
to me for a tutorial session.
My initial 5-10 minute tutorial starts off with an explanation of
name-binding, including something like "a=[1,2,3]; b=a; a[1]=4; print
b; b=42; print a, b". Note that this also demonstrates the dynamic
nature of Python typing -- 'b' is not restricted to being bound to a
list merely because the first time it was bound was to a list.
Once they have that down, I describe "immutable" vs. "mutable". I
start off by explaining that of course integers are immutable because
you cannot change 1 into 5. I then explain (with examples) that,
additionally, in the language, strings are immutable (which does NOT
mean you cannot usefully manipulate them, just that, for example, in "x
= 'abc'; x = x + 'd'" the manipulation "x + 'd'" results in a
completely new string. (The attempted example x[1] = "d" also gets them
used to the exception reporting mechanism.)
I actually spend more time (of this 10 minutes!) on "immutable" vs.
"mutable" (including ancillary lessons) than I spend on anything else.
After showing that strings are immutable, I then explain that tuples
are a sort of immutable list. This usually elicits the question of why
such a thing is needed, which is a great segue into a minute or so
spent on dictionaries and their keys. (Dictionaries can occupy
additional time if the student recognizes that they are a good fit for
a pressing need.)
Only about the last 30 seconds or so is spent on function calling. I
explain that all parameters are passed by reference, but that
OBVIOUSLY, if the called function wants to alter the caller's object,
the referenced object must be mutable, because, JUST AS WITH C
POINTERS, the pointer to the object is passed by making a copy of it on
the stack, so there is no way that the callee can directly alter the
caller's own pointer to the object.
After this lesson, I send them back to look at their script with
information about the online docs and a standing offer to help them
over any rough spots. NOT ONE PERSON has ever been confused about the
parameter passing.
I have only given these tutorials to accomplished programmers, so my
opinion about teaching newbies is not as well-formed, but for a
C/C++/assembler programmer, the REAL question about parameters is "What
is pushed onto the stack?" They are capable of a high level of
abstraction, so all they really want to know "Is a COPY of the object
passed on the stack, or is a POINTER to the object passed on the
stack?"
With such an accomplished student, if the teacher described Python's
parameter passing as anything but "call by reference", it would lead
to confusion, and, ultimately, disrespect for the teacher.
Regards,
Pat
ru...@yahoo.com
"Fredrik Lundh" <fre...@pythonware.com> wrote:
...snip...
> afaik, the Python Language Reference never defines the word "reference".
>
> It carefully defines words like "object" and "value", though, and terms like
> "call by object" or "call by object reference" are perfectly understandable
> if you use the words as they are defined in the language reference.
The Language Reference's definition of "value" is a non-definition.
About "value" it says that it is one of the three things an object
has. It then says some values can change and talks about
mutable and immutable. But we don't know what *it* is that
is mutable or immutable (other that it's called a "value" and it
is something an object "has").
Further down the page it says that container objects may have
references that are part of it's value. Part? Which part? Part
of what? Is an attribute part of a value?
I started a list of python doc problems a while ago. Right
near the top is "object value defintion is useless".
As I've said before, the Language Reference is badly
broken and needs a major rewrite. I don't think it should
be recommended to anyone as a source of authorative
information (except possibly for syntax since it in mostly
a Syntax Reference right now.)
[I posted another version of this yesterday from Google
but it has not appeared so...]
Antoon Pardon
> If we say "Python is call be reference" (or call by value, as many people
> also say) we *know* the consequence will be newbies writing in saying "I
> was told Python is call by reference, so I did this, and it didn't work,
> is that a bug in Python? What is wrong?" It is not a bug in Python, it is
> a bug in their mental model of how Python works, and we put that bug in
> their head. Every time that happens, it is *our* fault, not theirs, for
> using language guaranteed to mislead.
Well then I would think that the language that misleads is not
"reference" but assignment. Because it is not their idea of
call by reference that leads to the misconception, but their
idea of what an assignment does.
--
Antoon Pardon
ru...@yahoo.com
"Donn Cave" <do...@drizzle.com> wrote in message
news:11367924...@jetspin.drizzle.com...
Not yet. The reason is that I am still trying to figure out
what a value is myself. Do all objects have values? If
not which do and which don't? What's the value of int(1)?
An object? Some otherwise unreachable thing that
represents the abstract concept of the number 1?
What the value of object()? A few weeks ago I turned
to that page for enlightenment, with the results I reported.
> I find the topic difficult, myself. I think you really have to apply
> some context to the question, so there may not be any satisfactory
> definition for the language reference.
I have a hard time accepting that. I do not think there
is any aspect of human thought that cannot be described
by a sufficiently skilled writer.
> But maybe it would be simple with the right focus. If we could somehow
> define value, how would that help? I mean, presumably we need to
> understand all this stuff because we want to write some software, and
> if we dive in without understanding, our attempts will be plagued with
> conceptual errors. Is there something about value in particular that
> seems to be a problem here? ``No, you idiot, that's not a value -
> THIS is a value!''
Yes, see above. How can you feel confident working with
things that aren't understood? (c.f. this thead about
problems resulting from python beginner's misconceptions
about references.)
Bengt Richter
IMO the problem is that our intuitive notion of value is ususally really of
an abstract value, like an integer value or a real or maybe a member of a finite
abstract set. But in programming we are always dealing with various concrete
representations of values, not values themselves. To get to a value, you have
to interpret the representation. To perform abstract operations, you have to
manipulate concrete representations, and guarantee that the resulting representations
really do represent the abstract results of the abstract operation.
People are so used to interpreting representations that they do it unconsciously.
Mostly the metonymic use of representations in discussing abstractions is a
taken-for-granted short cut for avoiding tiresome pedantry, but sometimes, I believe,
it lies at the heart of difficulties such as in trying to define value.
IMO the first step in defining "value" should be to declare that it always refers
to some abstract value, which is always an interpretation of some concrete representation.
I suspect that describing things in terms of abstract graphs and their transformations
would be useful. E.g., starting with python's objects (in the abstract) as nodes,
and "references" as directed arcs.
More on this later, perhaps, but I have to go now ...
Regards,
Bengt Richter
Ben Sizer
Yes, my mistake - I forgot that Java reseats object references rather
than copying object values. (I'm a C++ person at heart.)
Personally I think this is a reason why Java is a poor language to
teach beginners, as it's stuck between low level semantics from C and
high level semantics like those of Python. It doesn't provide a very
consistent view, meaning you're bound to get confused no matter what
second language you start to learn.
--
Ben Sizer
Fredrik Lundh
> > afaik, the Python Language Reference never defines the word "reference".
> > It carefully defines words like "object" and "value", though, and terms like
> > "call by object" or "call by object reference" are perfectly understandable
> > if you use the words as they are defined in the language reference.
>
> It (sec. 3.1, "Objects, values and types") is not what I would
> call a good definition . About values it says only
>
> - that they are something that all objects have.
> - they can be mutable or immutable.
>
> It then has a few sentences about mutability, so after reading
> it you will know that, whatever a value is, it can be changed
> for some objects but not others. But what exactly it is that is
> being changed is still a mystery.
you really have trouble with abstract concepts, don't you?
*what* the value is is defined by the operations that the object supports (via its
type).
*how* the value is represented inside the object is completely irrelevant; a Python
implementation may use electric charges in small capacitors, piles of rocks, diapers,
or an endless supply of small guys in odd costumes to encode the value inside an
object. Changes to the value may be carried out by CPU instructions, caterpillars,
toddlers with gasmasks, or an endless supply of small guys in odd costumes. The
only thing that's important is that you can, in your Python program, access an ob-
jects value via its type, and get other objects back when you do.
</F>
Donn Cave
ru...@yahoo.com wrote:
> "Donn Cave" <do...@drizzle.com> wrote in message
> news:11367924...@jetspin.drizzle.com...
...
> > So you've had time to think about how you would define value, in a
> > few words. Any ideas?
>
> Not yet. The reason is that I am still trying to figure out
> what a value is myself. Do all objects have values? If
> not which do and which don't? What's the value of int(1)?
> An object? Some otherwise unreachable thing that
> represents the abstract concept of the number 1?
> What the value of object()? A few weeks ago I turned
> to that page for enlightenment, with the results I reported.
>
> > I find the topic difficult, myself. I think you really have to apply
> > some context to the question, so there may not be any satisfactory
> > definition for the language reference.
>
> I have a hard time accepting that. I do not think there
> is any aspect of human thought that cannot be described
> by a sufficiently skilled writer.
But you're asking for more than that. We're not just talking
about how people think about value, you want a definition that's
suitable for a language reference. Whereupon you would indeed
run into the kinds of questions you pose above, and more.
> > But maybe it would be simple with the right focus. If we could somehow
> > define value, how would that help? I mean, presumably we need to
> > understand all this stuff because we want to write some software, and
> > if we dive in without understanding, our attempts will be plagued with
> > conceptual errors. Is there something about value in particular that
> > seems to be a problem here? ``No, you idiot, that's not a value -
> > THIS is a value!''
>
> Yes, see above. How can you feel confident working with
> things that aren't understood? (c.f. this thead about
> problems resulting from python beginner's misconceptions
> about references.)
I'm saying that the definition of value doesn't contribute to
my understanding of my work. I guess we might say that the
whole point of a computer programming language is a mechanism
for the representation and manipulation of values, and our
task is to understand the mechanism enough to work with it.
That's what the language reference is for.
Donn Cave, do...@u.washington.edu
Paul Rubin
> But you're asking for more than that. We're not just talking
> about how people think about value, you want a definition that's
> suitable for a language reference. Whereupon you would indeed
> run into the kinds of questions you pose above, and more.
I know the Scheme standard includes a formal mathematical description
of the language semantics, maybe because the felt the usual verbiage
couldn't be precise enough.
ru...@yahoo.com
"Fredrik Lundh" <fre...@pythonware.com> wrote in message
news:mailman.272.11369167...@python.org...
> ru...@yahoo.com wrote:
>
> > > afaik, the Python Language Reference never defines the word "reference".
> > > It carefully defines words like "object" and "value", though, and terms like
> > > "call by object" or "call by object reference" are perfectly understandable
> > > if you use the words as they are defined in the language reference.
> >
> > It (sec. 3.1, "Objects, values and types") is not what I would
> > call a good definition . About values it says only
> >
> > - that they are something that all objects have.
> > - they can be mutable or immutable.
> >
> > It then has a few sentences about mutability, so after reading
> > it you will know that, whatever a value is, it can be changed
> > for some objects but not others. But what exactly it is that is
> > being changed is still a mystery.
>
> you really have trouble with abstract concepts, don't you?
Usually not when they are explained well.
> *what* the value is is defined by the operations that the object supports (via its
> type).
Well, that is already better than what is in the Lang Ref.
But there must be more to it than that. int(1) and int(2)
have exactly the same operations, yes? Yet their values
are different. (By "operations" you mean the set of methods
an object has?)
This interest is not just idle mental calesthenics.
I wanted to write a function that would dump the contents
of any object (value and attributes), and got rather confused
about values, types, repr's, etc.
It would help if you or someone would answer these
five questions (with something more than "yes" or "no" :-)
1. Do all objects have values?
2. What is the value of object()?
3. If two objects are equal with "==", does that
mean their values are the same?
4. Are object attributes part of an object's type
or it's value, or something else? (I think the first.)
5. The (only?) way to get an object's value is to
evaluate something (a name or a "reference"(*)
that refers to the object.
(*) I did not go back through this whole thread
but I know "reference" is controversial. I am not
trying to revive that debate. I mean the word in
a very generic sense.
> *how* the value is represented inside the object is completely irrelevant;
...snip...
Yes, I agree and did not intend to imply otherwise.
ru...@yahoo.com
I think the difference in our perspectives is that you already
*know* what a value is, not necessarily in a way that allows
you to write a defintion, but certainly in a way that allows
to work effectively with them.
As a Python beginner, I do not know, and I need something
more than "it is something an object has". I do NOT need
eiher some formal specifcation, nor a metaphysical discussion
that relates it to platonic ideals and other such concepts.
Surely there is some middle ground?
David Murmann
>> type).
>
> Well, that is already better than what is in the Lang Ref.
> But there must be more to it than that. int(1) and int(2)
> have exactly the same operations, yes? Yet their values
> are different. (By "operations" you mean the set of methods
> an object has?)
well, yes, but the behavior of the operations of an object are
based on its value. and in python you have no other possibility
to get closer to an objects value than to look what its methods
do (in the case of an int, print it, compare it, etc.). that
means the whole concept of a value is not really necessary to be
able to write a python program.
> 1. Do all objects have values?
> 2. What is the value of object()?
i'd say these two are closely related, and i'd say yes, all objects
have values and you can't really tell what the value of an object() is.
what you can say is, that two object()s have different values, because
their behavior differs (id(object)!).
but, one could define that id() is a special case and does not
contribute to the behavior of an object, so that different int
objects that print the same really have the same value.
> 3. If two objects are equal with "==", does that
> mean their values are the same?
no, absolutely not. think this class:
class C(object):
def __eq__(self, other):
return True
> 4. Are object attributes part of an object's type
> or it's value, or something else? (I think the first.)
they are part of an objects operations, just a shorthand for
__getattr__ and __setattr__.
> 5. The (only?) way to get an object's value is to
> evaluate something (a name or a "reference"(*)
> that refers to the object.
there is no such way (at least not to my knowledge).
at least not in general. of course you can print an int
and "see" its value, but you could define a C extension type
which holds a whole bunch of things as its value that are
completely inaccessible from python (are they then even part
of such an objects value? i don't care, its no use to define
"value" in either way).
i hope i did not confuse more than clarify, but this is the way
i see it.
also note that i understood Fredrik's quote from above in exactly
this way, this is just meant to clarify. (so please correct me if
i'm wrong)
--
David.
Steve Holden
>>
>>you really have trouble with abstract concepts, don't you?
>
>
> Usually not when they are explained well.
>
>
>>*what* the value is is defined by the operations that the object supports (via its
>>type).
>
>
> Well, that is already better than what is in the Lang Ref.
> But there must be more to it than that. int(1) and int(2)
> have exactly the same operations, yes? Yet their values
> are different. (By "operations" you mean the set of methods
> an object has?)
>
a car": a car is a complex object, with properties (number of doors, for
example, which can differ from car to car) and operations (like "start
engine", "accelerate", "brake" and so on).
> This interest is not just idle mental calesthenics.
> I wanted to write a function that would dump the contents
> of any object (value and attributes), and got rather confused
> about values, types, repr's, etc.
>
Well it's an ambitious project, but not an impossible one. Python has
features for "introspection", which means that programs can examine the
structure of the data rather than only dealing with values of known
structure.
> It would help if you or someone would answer these
> five questions (with something more than "yes" or "no" :-)
>
Well I've been staying away from this thread, but I'll try. For
demagogic purposes I am omitting consideration of what are usually
called "classic classes". You can talk about them once you understand
the situation in more detail.
Note to nitpickers
------------------
Please note that I *am* oversimplifying here, and the nitpickers will
undoubtedly find many threadsworth of valuable material here. The point
is to develop an understanding of the *principles*. Once someone has
that they can pick up the details as they need them, which 98% of Python
users don't feel the need to. I don't even claim to know the whole
story, but I *do* know enough to explain it in principle. So by all
means correct me where I am demonstrably wring, but *please* don't just
add complexity that doesn't aid understanding.
So sit comfortably, and we'll learn about object-oriented systems.
> 1. Do all objects have values?
All objects have a namespace. Each item in the namespace (attribute) is
identified by (guess what) a name, and refers to a value. The value is
also an object (which is why people sometimes say "in Python everything
is an object"). You might even argue that an object is nothing *but* a
namespace with specific behaviour, though that's a somewhat extreme view.
Each object is an instance of some type, and "knows" which type it's an
instance of. (A reference to) the type is usually stored in the
instance's attribute named (for historical reasons) __class__. Once you
get really smart you'll learn that you can even the type of some objects
(the ones whose types are defined in Python rather than being built into
the interpreter) by changing the value of their __class__ attribute.
The instance's type is *also* an object, and therefore has its own
namespace. The instance (of the type) is created by calling the type,
possibly with arguments that can be used to initialise the values of
items in the instance's namespace. They can also be ignored, but there
wouldn't be much point requiring them then, would there?
> 2. What is the value of object()?
Well, object is a type. So calling it give you an instance of type
object. But of course you could have answered this question for yourself
in the interactive interpreter:
>>> object()
<object object at 0x0099C438>
>>>
Pay attention here:
>>> object()
<object object at 0x0099C438>
No, I didn't just copy and paste the same text. I'm using the C Python
implementation, and the first time I created an instance of type object
I didn't bind it to a name. C Python keeps a count of references to all
objects, and when the reference count falls to zero it reclaims the
space. This was promptly re-used to create the second instance.
If I bind a name to a third object (which will *also* be created at the
same address) the memory won't be reused:
>>> a = object() # sets the ref count of the instance to 1
>>> object()
<object object at 0x0099C448>
>>>
> 3. If two objects are equal with "==", does that
> mean their values are the same?
Almost universally, yes, although if you know enough about how the
interpreter works "under the hood" you can define the response of
instances of your own classes to the "==" operator (by defining their
__eq__ method), and even define a class whose instances aren't equal to
anything, even to themselves!
> 4. Are object attributes part of an object's type
> or it's value, or something else? (I think the first.)
Well, this is where things become a little more complex.
Each instance of a type has some attributes defined in its own
namespace ("instance attributes"), which are therefore unique to it.
These attributes would normally be considered collectively as the
"value" of an object if you really *must* have something that you can
call the value. The attributes of an instance are accessible only by
referencing the instance and then qualifying that reference - usually
using the dot operator, but sometimes using functions like getattr().
You can, however, also use the instance to access the attributes of its
*type*, and perhaps this is what is confusing you. Because these type
attributes can be referenced via *any* instance of the type they are
considered common to all instances of a given type. Again for historical
reasons they are commonly called "class attributes".
The normal situation is that the instance attributes are used as the
instance's value, and the class attributes are used as the methods of
each instance of the type. This is why we say the type of an instance
defines its behaviour.
When you write
foo = instance.method(arg1, arg2)
you are almost always calling a function defined inside the instance
type's class definition (though this being Python there's absolutely
nothing to stop instances having callable attributes of their own too:
we are discussing "beginners' Python" here).
Under the hood the interpreter looks for an attribute called "method" in
the instance. Failing to find it, it then looks in the instance's type.
Of course it can fail to find it there, too. If the type is defined as a
specialisation of some other type (a "subclass" of the other type -
"type2 is like type1 but with the following differences") then the
interpreter will consult the other type, and so on and so on. I am
deliberately ignoring multiple inheritance ("typeC is like typeB, and
it's also like typeA") here, but the essentials are the same. We say the
subclass "inherits" the attributes of the superclass.
If the interpreter keeps walking up the inheritance tree in this way
without finding the attribute it's looking for, it will eventually
arrive at the type "object", because ultimately every object in Python
is defined as a subclass of (a subclass of (a subclass of ...)) object.
So, when you wrote above "... int(1) and int(2) have exactly the same
operations, yes? Yet their values are different" you were correct: the
operations are defined by their type ("int"), and are shared between all
instances. Unfortunately you chose an immutable type - an int object's
value cannot be changed, and so it isn't exposed to CPython's
introspection features. It's used inside the type's methods, which are
written in C and can therefore sneakily access bits of the objects that
don't live in the Python-accessible namespace.
> 5. The (only?) way to get an object's value is to
> evaluate something (a name or a "reference"(*)
> that refers to the object.
>
Well, most times you don't really get "an object's value", since the
value is collectively embedded within all of its attributes, and as
we've just seen, instances of immutable types don't expose their values
directly. But it's not *incorrect* to say that, just a bit fuzzy.
> (*) I did not go back through this whole thread
> but I know "reference" is controversial. I am not
> trying to revive that debate. I mean the word in
> a very generic sense.
>
>
>>*how* the value is represented inside the object is completely irrelevant;
>
> ...snip...
> Yes, I agree and did not intend to imply otherwise.
>
OK, so are you ready for some fun? The dir() builtin gives you access to
the names defined in an object's namespace. Let's go hunting.
>>> from pprint import pprint
This is just a convenience so we don't splurge across the page.
>>> pprint(dir(object))
['__class__',
'__delattr__',
'__doc__',
'__getattribute__',
'__hash__',
'__init__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__str__']
>>>
These are attributes that by definition every object must have (since
when told to look for them the interpreter will keep going if necessary
until it finds them in object). We can find some information out about
them by constructing a string referring to them and evaluating that
string (it's easier than typing them by hand into the interactive
interpreter).
>>> for name in dir(object):
... print name, ":", eval("object. %s" % name)
...
__class__ : <type 'type'>
__delattr__ : <slot wrapper '__delattr__' of 'object' objects>
__doc__ : The most base type
__getattribute__ : <slot wrapper '__getattribute__' of 'object' objects>
__hash__ : <slot wrapper '__hash__' of 'object' objects>
__init__ : <slot wrapper '__init__' of 'object' objects>
__new__ : <built-in method __new__ of type object at 0x1E1AE868>
__reduce__ : <method '__reduce__' of 'object' objects>
__reduce_ex__ : <method '__reduce_ex__' of 'object' objects>
__repr__ : <slot wrapper '__repr__' of 'object' objects>
__setattr__ : <slot wrapper '__setattr__' of 'object' objects>
__str__ : <slot wrapper '__str__' of 'object' objects>
>>>
This tells us that object's type is "type" (or as we'd say colloquially
"object is a type", just as we'd say "dict is a type"). Most of the
other attributes are methods or "slot wrappers". You can regard them as
the same for our purposes, as the difference is essentially
implementation detail. Now let's look at an *instance* of type object.
>>> b = object()
>>> for name in dir(b):
... print name, ":", eval("b. %s" % name)
...
__class__ : <type 'object'>
__delattr__ : <method-wrapper object at 0x00AC3CF0>
__doc__ : The most base type
__getattribute__ : <method-wrapper object at 0x00AC3E90>
__hash__ : <method-wrapper object at 0x00AC3CF0>
__init__ : <method-wrapper object at 0x00AC3D10>
__new__ : <built-in method __new__ of type object at 0x1E1AE868>
__reduce__ : <built-in method __reduce__ of object object at 0x0099C438>
__reduce_ex__ : <built-in method __reduce_ex__ of object object at
0x0099C438>
__repr__ : <method-wrapper object at 0x00AC3E90>
__setattr__ : <method-wrapper object at 0x00AC3D10>
__str__ : <method-wrapper object at 0x00AC3CF0>
>>>
Here you can see that the instance is actually *wrapping* it's type's
methods. Again this is an implementation detail: the point of the
wrapper is to make the C-implemented method look like a function defined
in Python, I believe - I wouldn't claim exact certainty on that.
You can see that the __new__ method is actually directly inherited from
the type (it has the same memory address), but all the other methods and
slot wrappers of the type appear to have been wrapped.
This isn't coincidental: __new__ is the only method that doesn't *need*
to be wrapped, because it's called to *create* the instance, and so it
gets passed a reference to the *type*, not the instance.
Now let's define our own type. Because "classic classes" continue to
exist, it's still necessary at present to explicitly state we want a
type. The simplest way to do this is to subclass object (because
otherwise we'd have to discuss metatypes, and your would end up
swallowing your brain). I'm just going to define one method for this type.
>>> class foo(object):
... "Pretty simple object subclass"
... def foo(self):
... print "foo.bar:", self
...
>>> for name in dir(foo):
... print name, ":", eval("foo.%s" % name)
...
__class__ : <type 'type'>
__delattr__ : <slot wrapper '__delattr__' of 'object' objects>
__dict__ : {'__dict__': <attribute '__dict__' of 'foo' objects>,
'__module__': '
__main__', 'foo': <function foo at 0x00B2FCF0>, '__weakref__':
<attribute '__weakref__' of 'foo' objects>, '__doc__': 'Pretty simple
object subclass'}
__doc__ : Pretty simple object subclass
__getattribute__ : <slot wrapper '__getattribute__' of 'object' objects>
__hash__ : <slot wrapper '__hash__' of 'object' objects>
__init__ : <slot wrapper '__init__' of 'object' objects>
__module__ : __main__
__new__ : <built-in method __new__ of type object at 0x1E1AE868>
__reduce__ : <method '__reduce__' of 'object' objects>
__reduce_ex__ : <method '__reduce_ex__' of 'object' objects>
__repr__ : <slot wrapper '__repr__' of 'object' objects>
__setattr__ : <slot wrapper '__setattr__' of 'object' objects>
__str__ : <slot wrapper '__str__' of 'object' objects>
__weakref__ : <attribute '__weakref__' of 'foo' objects>
foo : <unbound method foo.foo>
>>>
You can see what foos inherit from the "object" superclass: almost
everything; but the foo type also has a few of its own attributes:
__dict__, __weakref__, __module__, __doc__ and bar.
__dict__ is a dictionary with several keys in it: __dict__ ,
__weakref__, __module__, __doc__ and bar. Oh, those are the class
attributes that aren't inherited from the superclass!
Right, let's create a foo instance and see what that looks like.
>>> f = foo()
>>> for name in dir(f):
... print name, ":", eval("f.%s" % name)
...
__class__ : <class '__main__.foo'>
__delattr__ : <method-wrapper object at 0x00B36A10>
__dict__ : {}
__doc__ : Pretty simple object subclass
__getattribute__ : <method-wrapper object at 0x00B36A10>
__hash__ : <method-wrapper object at 0x00B369B0>
__init__ : <method-wrapper object at 0x00B36950>
__module__ : __main__
__new__ : <built-in method __new__ of type object at 0x1E1AE868>
__reduce__ : <built-in method __reduce__ of foo object at 0x00B36A30>
__reduce_ex__ : <built-in method __reduce_ex__ of foo object at 0x00B36A30>
__repr__ : <method-wrapper object at 0x00B36A50>
__setattr__ : <method-wrapper object at 0x00B36A10>
__str__ : <method-wrapper object at 0x00B369B0>
__weakref__ : None
foo : <bound method foo.foo of <__main__.foo object at 0x00B36A30>>
>>>
Again you can see that the instance wraps the methods of its type (even
the ones its type inherits from type "object"). The instance also has
its own (empty) __dict__ dictionary, and a "bound method". The binding
is actually created dynamically every time the method is referenced, and
the point is to attach a reference to the instance to a reference to the
method. This instance reference is passed as the value of the first
argument (usually called "self") when the method is called.
Now let's set an attribute of f and see what changes.
>>> f.attribute = "three"
>>> for name in dir(f):
... print name, ":", eval("f.%s" % name)
...
__class__ : <class '__main__.foo'>
__delattr__ : <method-wrapper object at 0x00B36950>
__dict__ : {'attribute': 'three'}
__doc__ : Pretty simple object subclass
__getattribute__ : <method-wrapper object at 0x00B36950>
__hash__ : <method-wrapper object at 0x00B36A10>
__init__ : <method-wrapper object at 0x00B369B0>
__module__ : __main__
__new__ : <built-in method __new__ of type object at 0x1E1AE868>
__reduce__ : <built-in method __reduce__ of foo object at 0x00B36A30>
__reduce_ex__ : <built-in method __reduce_ex__ of foo object at 0x00B36A30>
__repr__ : <method-wrapper object at 0x00B36930>
__setattr__ : <method-wrapper object at 0x00B36950>
__str__ : <method-wrapper object at 0x00B36A10>
__weakref__ : None
attribute : three
foo : <bound method foo.foo of <__main__.foo object at 0x00B36A30>>
>>>
Well, we can see that the new attribute is included in the result of the
dir() function. But if you look carefully you'll also see that *it's
been added to the instance's __dict__ dictionary*.
Read this over a few times and you should have a pretty good idea of
what goes where. Now get on with writing that function!
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/
Bengt Richter
[...]
>Note to nitpickers
>------------------
>Please note that I *am* oversimplifying here, and the nitpickers will
>undoubtedly find many threadsworth of valuable material here. The point
>is to develop an understanding of the *principles*. Once someone has
>that they can pick up the details as they need them, which 98% of Python
>users don't feel the need to. I don't even claim to know the whole
>story, but I *do* know enough to explain it in principle. So by all
>means correct me where I am demonstrably wring, but *please* don't just
>add complexity that doesn't aid understanding.
>When you write
>
> foo = instance.method(arg1, arg2)
>
>you are almost always calling a function defined inside the instance
>type's class definition (though this being Python there's absolutely
>nothing to stop instances having callable attributes of their own too:
>we are discussing "beginners' Python" here).
>
>Under the hood the interpreter looks for an attribute called "method" in
>the instance. Failing to find it, it then looks in the instance's type.
lookup before the mro logic. For new-style, the interpreter goes down the
type(instance).mro() chain looking for the attribute called "method"
until it finds a "method" attribute whose value has __get__ method AND a __set__ method
(which makes it a data descriptor, and not overridable by an instance attribute). If
found, it will call the method.__get__ method with the instance as an argument. The __get__ method
then returns (normally) the bound method. If it finds "method" without __get__ and __set__ methods,
it will ignore it, except that it may revisit it if after the whole mro chain fails, and there isn't
any "method" attribute on the instance either. In that case, it will look for "method" again along the
mro, and if it has a __get__ method, it will form a bound method (this is the typical "method" attribute
in the form of a function, which has a __get__ method), or if there is no __get__ method, it will
return the value as ordinary class variable. The re-scan of mro is not literal, hopefully. I'm just
trying to express the logic. Also special shortcuts apply for methods of builtins, I believe.
Old-style classes live alongside new ones -- I suspect -- by virtue of type(someinstance) being
<type 'instance'> and type(someinstance).__getattribute__ thus implementing old-style stuff instead
of the type(someinstance).__getattribute__ of new-style instances, which inherit their __getattribute__
either from object.__getattribute__ or type.__getattribute__, or a user definition.
>
>Of course it can fail to find it there, too. If the type is defined as a
>specialisation of some other type (a "subclass" of the other type -
>"type2 is like type1 but with the following differences") then the
>interpreter will consult the other type, and so on and so on. I am
>deliberately ignoring multiple inheritance ("typeC is like typeB, and
>it's also like typeA") here, but the essentials are the same. We say the
>subclass "inherits" the attributes of the superclass.
>
>If the interpreter keeps walking up the inheritance tree in this way
>without finding the attribute it's looking for, it will eventually
>arrive at the type "object", because ultimately every object in Python
>is defined as a subclass of (a subclass of (a subclass of ...)) object.
Unless the attribute access started on a type, in which case it will check
for type.method before object.method, UIAM. BTW, in order to find either
kind of attribute, there will first be a search for type(instance).__getattribute__,
which is found as object.__getattribute__ or type.__getattribute__ unless overridden
(or instance is of an old-style class as mentioned above).
I suspect these then implement the check for __getattr__, which can be user defined
to intercept attribute access on particular objects in the search chain, after
higher-priority stuff fails.
[... Can't ... keep .. eyes ... open ... for ... rest ...]
(not the fault of your prose, had a beer ;-)
Regards,
Bengt Richter
Bryan Olson
> The reason is that I am still trying to figure out
> what a value is myself. Do all objects have values?
Yes.
> If
> not which do and which don't? What's the value of int(1)?
> An object? Some otherwise unreachable thing that
> represents the abstract concept of the number 1?
The value is the integer one. Python's 'int' is an abstract
data type. It's values are the integers.
> What the value of object()? A few weeks ago I turned
> to that page for enlightenment, with the results I reported.
I think type 'object' has only one value, so that's it.
--
--Bryan
Mike Meyer
> ru...@yahoo.com wrote:
>> The reason is that I am still trying to figure out
>> what a value is myself. Do all objects have values?
> Yes.
Can you justify this, other than by quoting the manual whose problems
caused this question to be raised in the first place?
>> What the value of object()? A few weeks ago I turned
>> to that page for enlightenment, with the results I reported.
> I think type 'object' has only one value, so that's it.
In that case, they should all be equal, right?
>>> object() == object()
False
Looks like they have different values to me.
Or maybe an object is valueless, in spite of what the manual says.
Bryan Olson
> Bryan Olson writes:
>
>>ru...@yahoo.com wrote:
>>
>>>The reason is that I am still trying to figure out
>>>what a value is myself. Do all objects have values?
>>
>>Yes.
>
>
> Can you justify this, other than by quoting the manual whose problems
> caused this question to be raised in the first place?
The Python manual's claim there is solidly grounded. The logic
of 'types' is reasonably well-defined in the discipline. Each
instance of a type takes exactly one element from the type's
set of values (at least at a particular time).
>>>What the value of object()? A few weeks ago I turned
>>>to that page for enlightenment, with the results I reported.
>>
>>I think type 'object' has only one value, so that's it.
>
> In that case, they should all be equal, right?
>>>>object() == object()
>
> False
>
> Looks like they have different values to me.
Whether the '==' operation conforms to your idea of what equality
means is unclear. Maybe I was wrong, and the object's identity
is part of its abstract state.
> Or maybe an object is valueless, in spite of what the manual says.
We know that's not true.
--
--Bryan
Sybren Stuvel
>> I think type 'object' has only one value, so that's it.
>
> In that case, they should all be equal, right?
>
>>>> object() == object()
> False
You compare instances of the type 'object'. They both have one value:
>>> object()
<object object at 0xb7ddb438>
>>> object()
<object object at 0xb7ddb440>
So the claim "type 'object' has only one value" is true. It's just not
the same value for all instances.
Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
Fredrik Lundh
> Mike Meyer enlightened us with:
> >> I think type 'object' has only one value, so that's it.
> >
> > In that case, they should all be equal, right?
> >
> >>>> object() == object()
> > False
>
> You compare instances of the type 'object'. They both have one value:
>
> >>> object()
> <object object at 0xb7ddb438>
> >>> object()
> <object object at 0xb7ddb440>
>
> So the claim "type 'object' has only one value" is true. It's just not
> the same value for all instances.
You're comparing identities, not values. The value is the set of things that
you can access via an object's methods (via the type). The identity is not,
in itself, a part of the value.
Python doesn't query the object to determine it's type or identity, but it
always has to query the object to access the value.
A look at the C implementation of a typical object might help:
typedef struct {
int ob_refcnt;
struct _typeobject *ob_type; /* type */
... an unknown amount of stuff used to represent the value ...
} MyObject;
In CPython, the MyObject* pointer is the identity. The ob_refcnt field is a
CPython implementation detail. The ob_type field contains the type. The
rest of the structure is known only by the MyObject type implementation.
</F>
Donn Cave
ru...@yahoo.com wrote:
...
> I think the difference in our perspectives is that you already
> *know* what a value is, not necessarily in a way that allows
> you to write a defintion, but certainly in a way that allows
> to work effectively with them.
>
> As a Python beginner, I do not know, and I need something
> more than "it is something an object has". I do NOT need
> eiher some formal specifcation, nor a metaphysical discussion
> that relates it to platonic ideals and other such concepts.
>
> Surely there is some middle ground?
Well, see what you make of Steve Holden and Bengt Richter's
answers, and let us know if 1) it answers your question and
2) you can distill your new insight into a dozen words or so
for the language reference.
I'm still inclined to dispute the premise. Your example
where you demonstrate the problem:
> I wanted to write a function that would dump the contents
> of any object (value and attributes), and got rather confused
> about values, types, repr's, etc.
In the light of the answers you're getting, you may be thinking
that this isn't a simple problem. It isn't, in principle, it's
a huge bucket of worms. But if you have a practical focus that
comes out of your actual application for this function, it could
be pretty trivial. Your choice, and likewise for the notion of
value in general.
Donn Cave, do...@u.washington.edu
Bryan Olson
> Mike Meyer enlightened us with:
>
>>>I think type 'object' has only one value, so that's it.
>>
>>In that case, they should all be equal, right?
>>
>>
>>>>>object() == object()
>>
>>False
>
>
> You compare instances of the type 'object'. They both have one value:
>
>
>>>>object()
>
> <object object at 0xb7ddb438>
>
>>>>object()
>
> <object object at 0xb7ddb440>
>
> So the claim "type 'object' has only one value" is true. It's just not
> the same value for all instances.
No, that's not the issue. A type has a set of values (and a set of
operations); each instance takes one value from the type's set. I
think (I'm not sure) that object's set of values has only one element.
In Python, types are extensible, so by 'instance', I mean a direct
instance, not an instance of a class that inherits from 'object'.
Would it make sense to have a type with an empty set of values?
Sure. Such a type could never have a direct instance. Perhaps
'object' should be an abstract base class.
--
--Bryan
Steven D'Aprano
> It would help if you or someone would answer these
> five questions (with something more than "yes" or "no" :-)
>
> 1. Do all objects have values?
All objects ARE values. Some values themselves are complex objects
which in turn contain other values, e.g. if I execute:
L = [None, 1, "hello"]
I have created a name 'L' which is bound to ("has the value of") a list
with three items. The first item has the value of ("is the object") None,
the second has the value of ("is the object") 1, and the third is the
string "hello".
> 2. What is the value of object()?
These seems complicated because we are using the same word ("object") for
both the generic programming term, the class (strictly type, although the
difference is not important), and the instance itself.
Let's make it less confusing by using a case where the three things are
different:
class Foo:
pass
Foo is a class. Classes are (generic programming term) objects.
Foo() returns an instance of the Foo class. Instances are (generic
programming term) objects.
The value of Foo() is the instance. Why? int(data) returns an instance of
int, and str(data) returns an instance of str. It makes sense to say that
the value of the int instance 1 is one, the value of str instance "abc" is
the string "abc", and so forth. So we'd like to talk the same way about
more complex objects: the value of an instance is itself.
Hence the value of Foo() is the Foo instance at some particular memory
location. In this specific case, Foo() creates an object with very little
behaviour and and no distinguishing characteristics except their location
in memory, where as ints like 1 2 and 7 are easily distinguished.
Now let's return to (non-generic) object(). object is (as well as the
generic programming term) a class (strictly type, but the difference is
unimportant). object() returns an instance of class object:
>>> object()
<object object at 0xf705e3b8>
So the value of object() is some particular instance (of type object).
> 3. If two objects are equal with "==", does that
> mean their values are the same?
Intuitively yes, but not necessarily:
>>> 3.0 == 3
True
(We want the float 3.0 to be equal to the int 3, but we can distinguish
between them by something more than their location in memory.)
You can define a custom class:
class EqualAll:
def __eq__(self, other):
return True
but that's a pathological case.
> 4. Are object attributes part of an object's type
> or it's value, or something else? (I think the first.)
The type/class defines what attributes (generic) objects have. The value
of the attribute lives with the instance itself (that's why fred.attribute
and barney.attribute can be different). Whether you want to say the
attribute itself is part of the class or part of the instance value is, in
my opinion, not a useful question. The answer depends on which way you
want to look at it.
> 5. The (only?) way to get an object's value is to
> evaluate something (a name or a "reference"(*)
> that refers to the object.
I can't think of any other way to get at an object except by accessing
that object, or even what it would mean if there was such a way.
--
Steven.
Donn Cave
| On Thu, 12 Jan 2006 16:11:53 -0800, rurpy wrote:
|> It would help if you or someone would answer these
|> five questions (with something more than "yes" or "no" :-)
|>
|> 1. Do all objects have values?
|> 2. What is the value of object()?
[ I assume you mean, the object returned by object(). ]
It doesn't really have a value. I can't think of any kind of
computation that could use this object directly.
|> 3. If two objects are equal with "==", does that
|> mean their values are the same?
Yes.
| >>> 3.0 == 3
| True
Evidently the value of 3.0 is the same as the value of 3.
|> 4. Are object attributes part of an object's type
|> or it's value, or something else? (I think the first.)
|
| The type/class defines what attributes (generic) objects have. The value
| of the attribute lives with the instance itself (that's why fred.attribute
| and barney.attribute can be different).
I think to be strictly accurate, attributes and their values may reside
in a class instance, or in (one of) its class(es.) In the most common
case, functions will be in the class and data will be in the instance,
but every variation on this is reasonably common.
| ... Whether you want to say the
| attribute itself is part of the class or part of the instance value is, in
| my opinion, not a useful question. The answer depends on which way you
| want to look at it.
For a given class and instance, it can be more obvious. Take the object
returned by object(), which has attributes like '__reduce__' - all those
attributes may reasonably be considered "type" and not "value".
On the other hand, there are no constraints on how you may use these
namespaces. You can use an instance, or a class, like you would use
a dictionary object, and then it's all value.
Donn Cave, do...@drizzle.com
Steven D'Aprano
> Quoth Steven D'Aprano <st...@REMOVETHIScyber.com.au>:
> | On Thu, 12 Jan 2006 16:11:53 -0800, rurpy wrote:
> |> It would help if you or someone would answer these
> |> five questions (with something more than "yes" or "no" :-)
> |>
> |> 1. Do all objects have values?
> ...
> |> 2. What is the value of object()?
>
> [ I assume you mean, the object returned by object(). ]
>
> It doesn't really have a value. I can't think of any kind of
> computation that could use this object directly.
Here is one:
obj_inst = object()
def computation(data):
global obj_inst
if data is obj_inst:
print FirstOneThousandPrimes()
else:
print TextOfWarAndPeace()
It isn't a particularly useful computation, but it is a computation.
> |> 3. If two objects are equal with "==", does that
> |> mean their values are the same?
>
> Yes.
>
> | >>> 3.0 == 3
> | True
>
> Evidently the value of 3.0 is the same as the value of 3.
Okay.
>>> "3" == 3
False
Evidently the value of three is not the same as the value of three.
Let's try not to be too deep here, okay? Before asking "what is the value
of foo?", we have to agree on what we mean by "value". It is easy to tie
yourself into knots here.
> |> 4. Are object attributes part of an object's type
> |> or it's value, or something else? (I think the first.)
> |
> | The type/class defines what attributes (generic) objects have. The value
> | of the attribute lives with the instance itself (that's why fred.attribute
> | and barney.attribute can be different).
>
> I think to be strictly accurate, attributes and their values may reside
> in a class instance, or in (one of) its class(es.)
When you call instance.attribute, attribute may be either a class
attribute or an instance attribute. If it is a class attribute, it belongs
to the class, not the instance, even if you access it through the
attribute.
> In the most common
> case, functions will be in the class and data will be in the instance,
> but every variation on this is reasonably common.
> | ... Whether you want to say the
> | attribute itself is part of the class or part of the instance value is, in
> | my opinion, not a useful question. The answer depends on which way you
> | want to look at it.
>
> For a given class and instance, it can be more obvious. Take the object
> returned by object(), which has attributes like '__reduce__' - all those
> attributes may reasonably be considered "type" and not "value".
I don't see that "value" or "type" are opposite, I see that they are
orthogonal. The type of 1 is int, the value of 1 is integer one. The
type of object() is object, the value is itself. The type of
object().__reduce__ is method, and the value of object().__reduce__ is
the particular (generic) object bound to the name __reduce__ in the class
object namespace. That method is a class attribute, and so it part of the
value of object().
Am I the only one that wishes that object() had been given another name,
so it would be easier to talk about the generic programming concept object
and the specific Python new-style class object without confusion?
--
Steven.
Bryan Olson
> You're comparing identities, not values. The value is the set of things that
> you can access via an object's methods (via the type).
Which does make '==' kind of weird. It may or may not refer to
a method of the object.
> The identity is not,
> in itself, a part of the value.
>
> Python doesn't query the object to determine it's type or identity, but it
> always has to query the object to access the value.
>
> A look at the C implementation of a typical object might help:
>
> typedef struct {
> int ob_refcnt;
> struct _typeobject *ob_type; /* type */
> ... an unknown amount of stuff used to represent the value ...
> } MyObject;
>
> In CPython, the MyObject* pointer is the identity. The ob_refcnt field is a
> CPython implementation detail. The ob_type field contains the type.
So, was it an editing error when you said that Python does not
query the object to determine its type? The type is there in the
object, and and in Python, variables and references are not typed.
--
--Bryan
Fredrik Lundh
> > The identity is not, in itself, a part of the value.
> >
> > Python doesn't query the object to determine it's type or identity, but it
> > always has to query the object to access the value.
> >
> > A look at the C implementation of a typical object might help:
> >
> > typedef struct {
> > int ob_refcnt;
> > struct _typeobject *ob_type; /* type */
> > ... an unknown amount of stuff used to represent the value ...
> > } MyObject;
> >
> > In CPython, the MyObject* pointer is the identity. The ob_refcnt field is a
> > CPython implementation detail. The ob_type field contains the type.
>
> So, was it an editing error when you said that Python does not
> query the object to determine its type? The type is there in the
> object, and and in Python, variables and references are not typed.
do you want to understand this, or do you just want to argue ?
(hint: what might the word "query" mean in this context? can you think
of a meaning that's compatible with what I wrote? would concepts like
"object", "value", "type", and "identity" be radically different on a python
implementation that used e.g. "true" garbage collection and tagged pointers
instead of CPython's approach? if so, why?)
</F>