Counterintuitive Python behavior
domin...@yahoo.com
one thing I like very much about Python is that statements
work like you would expect them to work. Take for example
the use of dict.values() for dictionaries: If you store the
result of dict.values(), and change the dictionary after-
wards, the previously stored result remains untouched.
>>> dict = {'a':1,'b':2}
>>> list = dict.values()
>>> list
[1, 2]
>>> dict['a']=3
>>> list
[1, 2]
>>> dict
{'a': 3, 'b': 2}
However, if a dictionary has lists as value entries, I get
a counterintuitive behavior (which, just recently, broke my
code): If you change the dict, the list you previously
created via dict.values() gets automagically updated. A nice
feature, but nothing I would have expected!
>>> dict = {'a':[1],'b':[2]}
>>> list = dict.values()
>>> list
[[1], [2]]
>>> dict['a'].append(3)
>>> dict
{'a': [1, 3], 'b': [2]}
>>> list
[[1, 3], [2]]
Looks like that in the first case a copy is returned while
in the latter case list references are returned. Ok, but
according to Python's philosophy I shouldn't mind if I work
with lists in the dictionary or anything else. If the
behavior depends on the knowledge of the type of values I
put into a dictionary, I find that somehow counterintuitive.
Who is wrong here: my intuition or Python (2.2)? If it's
my intuition, how can I train my thinking about Python's
execution model, so that my intuition get's better ;-)
Best regards,
Dominikus
Max M
> Who is wrong here: my intuition or Python (2.2)? If it's
> my intuition, how can I train my thinking about Python's
> execution model, so that my intuition get's better ;-)
Your intuition!
Objects like lists gets copied by reference. So Python behaves exactly
like it should.
Maybe it is clearer with another notation:
>>> dict = {'a':<list ref1>,'b':<list ref2>}
>>> list = dict.values()
>>> list
[<list ref1>, <list ref2>]
>>> dict['a'].append(3)
>>> dict
{'a': <list ref1>, 'b': <list ref2>}
>>> list
[<list ref1>, <list ref2>]
regards Max M
Michael Hudson
> one thing I like very much about Python is that statements
> work like you would expect them to work.
Well, Python works very much as I expect it, but it's not clear if
this says more about me or Python <wink>.
At the end of your email, you say:
> Who is wrong here: my intuition or Python (2.2)? If it's
> my intuition, how can I train my thinking about Python's
> execution model, so that my intuition get's better ;-)
It's you :) As I can't read my email at the moment[1], I have no
better way of wasting my time to hand than drawing you some ascii art.
First, some terminology. Actually, the very first thing is some
anti-terminology; I find the word "variable" to be particularly
uphelpful in a Python context. I prefer "names", "bindings" and
"objects".
Names look like this:
,-----.
| foo |
`-----'
Names live in namespaces, but that's not really important for the
matter at hand as the only namespace in play is the one associated
with the read-eval-print loop of the interpreter. In fact names are
only minor players in the current drama; bindings and objects are the
real stars.
Bindings look like this:
------------>
Bindings' left ends can be attached to names, or other "places" such
as attributes of objects and entries in lists or dictionaries. Their
right hand ends are always attached to objects[2].
Objects look like this:
+-------+
| "bar" |
+-------+
This is meant to be the string "bar". Other types of object will be
drawn differently, but I hope you'll work out what I'm up to.
> Take for example the use of dict.values() for dictionaries: If you
> store the result of dict.values(), and change the dictionary after-
> wards, the previously stored result remains untouched.
>
> >>> dict = {'a':1,'b':2}
After this statement, it would seem appropriate to draw this picture:
,------. +-------+
| dict |------>|+-----+| +---+
`------' || "a" |+---->| 1 |
|+-----+| +---+
|+-----+| +---+
|| "b" |+---->| 2 |
|+-----+| +---+
+-------+
> >>> list = dict.values()
Now this:
,------. +-------+
| dict |------>|+-----+| +---+
`------' || "a" |+------------>| 1 |
|+-----+| +---+
|+-----+| /\
|| "b" |+-----. ,---'
|+-----+| | |
+-------+ `----+----.
| |
,------. +-----+ | \/
| list |------>| [0]-+------------' +---+
`------' | [1]-+--------------->| 2 |
+-----+ +---+
> >>> list
> [1, 2]
Which is of course, no surprise.
> >>> dict['a']=3
Now this:
,------. +-------+
| dict |------>|+-----+| +---+
`------' || "a" |+-. | 1 |
|+-----+| | +---+
|+-----+| | /\
|| "b" |+-+---. ,---'
|+-----+| | | |
+-------+ | `----+----.
| | |
,------. +-----+ | | \/
| list |------>| [0]-+---+--------' +---+
`------' | [1]-+---+----------->| 2 |
+-----+ | +---+
| +---+
`----------->| 3 |
+---+
> >>> list
> [1, 2]
> >>> dict
> {'a': 3, 'b': 2}
These should also come as no surprise; just chase the arrows
(bindings) above.
> However, if a dictionary has lists as value entries, I get
> a counterintuitive behavior (which, just recently, broke my
> code): If you change the dict, the list you previously
> created via dict.values() gets automagically updated. A nice
> feature, but nothing I would have expected!
That's because you're not thinking in terms of Names, Objects and
Bindings.
> >>> dict = {'a':[1],'b':[2]}
,------. +-------+
| dict |------>|+-----+| +-----+ +---+
`------' || "a" |+---->| [0]-+-->| 1 |
|+-----+| +-----+ +---+
|+-----+| +-----+ +---+
|| "b" |+---->| [0]-+-->| 2 |
|+-----+| +-----+ +---+
+-------+
> >>> list = dict.values()
,------. +-------+
| dict |------>|+-----+| +-----+ +---+
`------' || "a" |+------------>| [0]-+-->| 1 |
|+-----+| +-----+ +---+
|+-----+| /\
|| "b" |+-----. ,----'
|+-----+| | |
+-------+ `----+-----.
| |
,------. +-----+ | \/
| list |------>| [0]-+------------' +-----+ +---+
`------' | [1]-+--------------->| [0]-+-->| 2 |
+-----+ +-----+ +---+
> >>> list
> [[1], [2]]
Again, no surprises here.
> >>> dict['a'].append(3)
+---+
,------. +-------+ ,->| 1 |
| dict |------>|+-----+| +-----+ | +---+
`------' || "a" |+------------>| [0]-+-'
|+-----+| | [1]-+-.
|+-----+| +-----+ | +---+
|| "b" |+-----. /\ `->| 3 |
|+-----+| | ,----' +---+
+-------+ | |
`----+-----.
,------. +-----+ | \/
| list |------>| [0]-+------------' +-----+ +---+
`------' | [1]-+--------------->| [0]-+-->| 2 |
+-----+ +-----+ +---+
> >>> dict
> {'a': [1, 3], 'b': [2]}
> >>> list
> [[1, 3], [2]]
And now these should not be surprising either.
> Looks like that in the first case a copy is returned while
> in the latter case list references are returned. Ok, but
> according to Python's philosophy I shouldn't mind if I work
> with lists in the dictionary or anything else. If the
> behavior depends on the knowledge of the type of values I
> put into a dictionary, I find that somehow counterintuitive.
If you haven't realised where you're misconceptions come from from the
above pictures, I'm not sure more prose would help.
Cheers,
M.
[1] Does anyone know where the starship's gone?
[2] Anyone mentioning UnboundLocalError at this point will be shot.
--
A.D. 1517: Martin Luther nails his 95 Theses to the church door and
is promptly moderated down to (-1, Flamebait).
-- http://slashdot.org/comments.pl?sid=01/02/09/1815221&cid=52
(although I've seen it before)
Alex Martelli
> Hello,
>
> one thing I like very much about Python is that statements
> work like you would expect them to work. Take for example
> the use of dict.values() for dictionaries: If you store the
> result of dict.values(), and change the dictionary after-
> wards, the previously stored result remains untouched.
The .values() method of a dictionary is defined to return
a new list of the values. That's more or less inevitable,
since a dictionary doesn't _have_ a list of its value
normally, so it must build it on the fly when you ask for it.
It's not a copy -- it's a new list object.
However, Python does NOT copy except in situations where
a copy is specifically defined to happen. The .values()
method being in a vague sense such a situation, as mentioned...
a new object, rather than a copy of any existing one.
In general, whenever possible, Python returns references
to the same objects it already had around, rather than
copying; if you DO want a copy you ask for it -- see module
copy if you want to do so in a general way. Of course,
building new objects is a different case.
If this is counteintuitive, so be it -- there is really
no alternative in the general case without imposing huge
overhead, making copies of everything "just in case".
MUCH better to get copies only on explicit request (and
new objects, when there's no existing object that could
either be copied or referred-to).
Of course there are in-between cases -- such as slices.
The standard sequences give you a new object when you
ask for a slice; this only matters for lists (for immutable
objects you shouldn't care if you get copies or what).
A list is not able to "share a part of itself", so when
asked for a slice it gives out a copy, a new list (for
generality, of course, it then also does when asked for
a slice-of-everything, thelist[:] -- so in that limit case
the new object can be seen as a copy of the existing one).
The justly popular Numeric package, on the other hand,
defines an array type which IS able to share some or all
data among several array objects -- so a slice of a Numeric
array does share data with the array it's sliced from. It's
a new object, mind you:
>>> import Numeric
>>> a=Numeric.array(range(6))
>>> b=a[:]
>>> id(a)
136052568
>>> id(b)
136052728
>>>
but the two distinct objects a and b do share data:
>>> a
array([0, 1, 2, 3, 4, 5])
>>> b
array([0, 1, 2, 3, 4, 5])
>>> a[3]=23
>>> b
array([ 0, 1, 2, 23, 4, 5])
>>>
Each behavior has excellent pragmatics behind it -- lists
are _way_ simpler by not having to worry about data sharing,
arrays have different use cases by far -- but it's hard to
be unsurprising when two somewhat similar objects differ
in such details.
But all of the copies which do "happen", e.g. by the
limit case of list slicing or whatever else (with ONE
exception of which more later) are always SHALLOW copies.
NEVER does Python embark on the HUGE task of _deep_ copying
unless you very specifically ask it to -- specifically with
function deepcopy of module copy. DEEP copying is a serious
matter -- function deepcopy has to watch out for cycles,
reproduce any identity of references, potentially follow
references to any depth, recursively -- it has to reproduce
faithfully a graph of objects referencing each other
with unbounded complexity. It works, but of course it
can never be as fast as the mundane business of shallow
copying (which in turn is never as fast as just handing
out one more reference to an existing object, whenever
the latter course of action is feasible).
So, that's what has apparently snagged you here:
> However, if a dictionary has lists as value entries, I get
> a counterintuitive behavior (which, just recently, broke my
> code): If you change the dict, the list you previously
> created via dict.values() gets automagically updated. A nice
> feature, but nothing I would have expected!
Not really -- if you change _objects to which the dict refers_
(rather than changing the dict in se), then other references
to just-the-same-objects remain references to just the same
objects -- if the objects mutate, you see the mutated objects
from whatever references to them you may be using.
>>>> dict = {'a':[1],'b':[2]}
>>>> list = dict.values()
>>>> list
> [[1], [2]]
Don't use the names of built-in types as variables: you WILL
be burned one day if you do this. dict, list, str, tuple, file,
int, long, float, unicode... do NOT use these identifiers for
your own purposes, tempting though they may be (an "attractive
nuisance", to be sure). If you don't get into the habit of
avoiding them, one day you'll be trying to (e.g.) build a
list with x=list('ciao') and get puzzling errors... because
you have rebound identifier 'list' to refer to a certain list
object rather than to the list type itself.
Use alist, somedict, myfile, whatever... nothing to do with
your problem here, just some other simple advice!-)
>>>> dict['a'].append(3)
This does not "change the dictionary" -- the dictionary object
still contains exactly the same references, to objects with
the same id's (two string objects, the keys, and two list
objects, the values). You're changing (mutating) one of those
objects, but that's quite another issue. You could be
mutating said list object through any reference to it
whatsoever, e.g.:
>>> alist=list('ciao')
>>> adict={'a':alist}
>>> adict
{'a': ['c', 'i', 'a', 'o']}
>>> alist.pop()
'o'
>>> adict
{'a': ['c', 'i', 'a']}
>>>
If you wanted dictionary adict to refer to a COPY (a "snapshot",
if you will) of the contents of alist, you could have said so:
>>> import copy
>>> alist=list('ciao')
>>> adict={'a':copy.copy(alist)}
>>> adict
{'a': ['c', 'i', 'a', 'o']}
>>> alist.pop()
'o'
>>> adict
{'a': ['c', 'i', 'a', 'o']}
>>>
and then the dictionary object's string-representation would
be isolated from whatever changes to the list to which name
alist refers. The string representation delegates part of its
job to the objects to which the dictionary object refers, so,
if you want to isolate it, you do need copies -- maybe deep
ones, in fact (<shudder>... well no, not really, but...:-).
>>>> dict
> {'a': [1, 3], 'b': [2]}
>>>> list
> [[1, 3], [2]]
>
> Looks like that in the first case a copy is returned while
> in the latter case list references are returned. Ok, but
Nope. ALWAYS references. .values() doesn't return a reference
to an existing object NOR a copy of an existing object, because
there's no "existing object" in this case -- so it always
returns a NEW object, suitably built as per its specs.
> according to Python's philosophy I shouldn't mind if I work
> with lists in the dictionary or anything else. If the
> behavior depends on the knowledge of the type of values I
> put into a dictionary, I find that somehow counterintuitive.
There is no such dependence. Just a huge difference
between changing an object, and changing (mutating) some
OTHER object to which the first refers.
In Bologna over 100 years ago we had a statue of a local hero
depicted pointing forwards with his finger -- presumably to
the future, but given where exactly it was placed, the locals
soon identified it as "the statue that points to Hotel
Belfiore". The one day some enterprising developer bought
the hotel's building and restructured it -- in particular,
where the hotel used to be was now a restaurant, Da Carlo.
So, "the statue that points to Hotel Belfiore" had suddenly
become "the statue that points to Da Carlo"...! Amazing
isn't it? Considering that marble isn't very fluid and the
statue had not been moved or disturbed in any way...?
This is a real anecdote, by the way (except that I'm not
sure of the names of the hotel and restaurant involved --
I could be wrong on those), but I think it can still help
here. The dictionary, or statue, has not changed at all,
even though the objects it refers/points to may have been
mutated beyond recognition, and the name people know it
by (the dictionary's string-representation) may therefore
change. That name or representation was and is referring
to a non-intrinsic, non-persistent, "happenstance"
characteristic of the statue, or dictionary...
> Who is wrong here: my intuition or Python (2.2)? If it's
> my intuition, how can I train my thinking about Python's
> execution model, so that my intuition get's better ;-)
Your intuition, which led you astray here (Python does just
what it should do), can be trained in several ways. The
works of J. L. Borges and I. Calvino, if you like fiction
that's reasonably sophisticated but still quite pleasant,
are good bets. If you like non-fiction written by
engineers fighting hard to dispell some of the errors of
philosophers, Wittgenstein and Korzibsky are excellent.
I'm not kidding, but I realize that many Pythonistas don't
really care for either genre. In which case, this group
and its archives, essays by GvR and /F, and the Python
sources, may also prove interesting reading.
Alex
domin...@yahoo.com
I understand that! But my main objection remains: Why should
I care about the type of object being handled by a dictionary?
This could not only break unit tests but also make maintenance
of ``old'' code somewhat painful.
Regards,
Dominikus
Sean 'Shaleh' Perry
On 17-Apr-2002 Max M wrote:
> domin...@yahoo.com wrote:
>
>> Who is wrong here: my intuition or Python (2.2)? If it's
>> my intuition, how can I train my thinking about Python's
>> execution model, so that my intuition get's better ;-)
>
> Your intuition!
>
> Objects like lists gets copied by reference. So Python behaves exactly
> like it should.
>
Or from another side:
a = [...]
b = a
b[n] = ...
and a gets updated. values() returns the contents of the dictionary so you are
seeing the same type of behaviour.
bru...@tbye.com
> It's you :) As I can't read my email at the moment[1], I have no
> better way of wasting my time to hand than drawing you some ascii art.
[snip]
This is a great explanation! The next time a co-worker asks a
assignment/binding/references question I'm going to refer them to this
message. Thanks for writing this!
[snip]
> +---+
> ,------. +-------+ ,->| 1 |
> | dict |------>|+-----+| +-----+ | +---+
> `------' || "a" |+------------>| [0]-+-'
> |+-----+| | [1]-+-.
> |+-----+| +-----+ | +---+
> || "b" |+-----. /\ `->| 3 |
> |+-----+| | ,----' +---+
> +-------+ | |
> `----+-----.
> ,------. +-----+ | \/
> | list |------>| [0]-+------------' +-----+ +---+
> `------' | [1]-+--------------->| [0]-+-->| 2 |
> +-----+ +-----+ +---+
(sorry for reposting this, but it was too cool to snip)
-Dave
Alex Martelli
...
> I understand that! But my main objection remains: Why should
> I care about the type of object being handled by a dictionary?
You shouldn't: whatever types of objects are the dictionary's values,
method .values() will ALWAYS return a new list object whose items
refer to EXACTLY the same objects as the dictionary's values.
> This could not only break unit tests but also make maintenance
> of ``old'' code somewhat painful.
I don't understand what you're talking about. My sentence above
was true at least since Python 1.5.2 (probably earlier, but that's
when I hopped aboard).
Alex
domin...@yahoo.com
you are excellent teachers!!!
Michael, you helped me really getting the point with your
drawings. Thanks a lot for your art work!
Alex, the anecdote about the statue pointing to Hotel Belfiore
made my wrong intuition so obvious! I like that and will never
ever forget it anymore! Thanks for your answer!
I think, today I learnt a lot on my way becoming a real Pythoniac!
Best regards,
Dominikus
domin...@yahoo.com
> > I care about the type of object being handled by a dictionary?
>
> You shouldn't: whatever types of objects are the dictionary's values,
> method .values() will ALWAYS return a new list object whose items
> refer to EXACTLY the same objects as the dictionary's values.
I got it after I read your and Michaels posts!
> > This could not only break unit tests but also make maintenance
> > of ``old'' code somewhat painful.
>
> I don't understand what you're talking about. My sentence above
> was true at least since Python 1.5.2 (probably earlier, but that's
> when I hopped aboard).
>
> Alex
Now, I don't understand it myself anymore ;-)
Dominikus
Michael Hudson
> On Wed, 17 Apr 2002, Michael Hudson wrote:
>
> > It's you :) As I can't read my email at the moment[1], I have no
> > better way of wasting my time to hand than drawing you some ascii art.
> [snip]
>
> This is a great explanation!
Thanks!
> The next time a co-worker asks a
> assignment/binding/references question I'm going to refer them to this
> message.
I was hoping more people than the OP would find it useful. I'll
probably knock up a version for my web pages (when the starship
reappears...).
Cheers,
M.
--
People think I'm a nice guy, and the fact is that I'm a scheming,
conniving bastard who doesn't care for any hurt feelings or lost
hours of work if it just results in what I consider to be a better
system. -- Linus Torvalds
Michael Hudson
> > The next time a co-worker asks a assignment/binding/references
> > question I'm going to refer them to this message.
>
> I was hoping more people than the OP would find it useful. I'll
> probably knock up a version for my web pages (when the starship
> reappears...).
I should say that Alex's answer is well worth referring to too. I
wonder if he would mind me incorporating it into aforementioned
webpage...
Cheers,
M.
--
If a train station is a place where a train stops, what's a
workstation? -- unknown (to me, at least)
Terry Reedy
<domin...@yahoo.com> wrote in message
news:3CBD1C02...@yahoo.com...
Preliminary note: over-riding the builtin constructor function names
'list' and 'dict' with program values is a bad habit to get into. So
I modified the quotes.
vlist = ddict.values()
...
> However, if a dictionary has lists as value entries, I get
> a counterintuitive behavior (which, just recently, broke my
> code): If you change the dict, the list you previously
> created via ddict.values() gets automagically updated
The only list created by ddict.values() is the list of references to
the values in ddict.
> >>> ddict['a']=3
This changes the reference to 2 to a new reference to 3.
ddict.values() is changed and now different from vlist.
> >>> ddict['a'].append(3)
This does not change the reference bound to a and hence ddict.values()
is unchanged and still equal to vlist. It uses the reference bound to
'a' to access and change the list, but *this* change is invisible to
ddict.
Terry J. Reedy
Alex Martelli
> Michael Hudson <m...@python.net> writes:
>
>> > The next time a co-worker asks a assignment/binding/references
>> > question I'm going to refer them to this message.
>>
>> I was hoping more people than the OP would find it useful. I'll
>> probably knock up a version for my web pages (when the starship
>> reappears...).
>
> I should say that Alex's answer is well worth referring to too. I
> wonder if he would mind me incorporating it into aforementioned
> webpage...
Sure! Go right ahead.
Alex
Greg Weeks
: >>> dict = {'a':[1],'b':[2]}
: >>> list = dict.values()
: >>> dict['a'].append(3)
: >>> list
: [[1, 3], [2]]
: Who is wrong here: my intuition or Python (2.2)?
As others have said, Python is right on this. However, Python aids you in
your confusion, as follows:
There should have been an operator in Python that says when objects are
conceptually the same. For the moment, call it "eq". The following would
hold:
3 eq 3
not (3 eq 4)
a eq a # PROVIDED a HAS A VALUE
"foo" eq "foo"
not ([] eq [])
not ({} eq {})
() eq ()
Furthermore, the behavior of "eq" for class instances should be user
controllable. Typically, immutable objects would be "eq" when their parts
are "eq", while mutable objects would be "eq" only when one "is" the other.
Furthermore, two "eq" objects should hash to the same value. So *any*
object could be a hash key.
Python almost has an "eq" operator. "==" is like "eq" with two exceptions:
lists
dictionaries
Since "==" is so close to "eq", it is natural to think of "==" as meaning
conceptual sameness. And if "==" meant conceptual sameness, then lists
would be immutable, similar to Lisp lists (in the absence of rplaca and
rplacd). In that case, .append(3) would have been implemented by consing,
and your intuition would have been right.
I'm not saying that your mind did work this way. But it might have.
Greg
PS: Interestingly enough, all the expressions involving "eq" above are true
on my system with "eq" replaced by "is". But (I believe) the following are
implementation-dependent:
3 eq 3
"foo" eq "foo"
() eq ()
And "eq" disagrees with "is" for (properly defined) immutable class
instances. And, although I didn't mention it above, 3 eq 3.0 should
probably hold. (Which, I should add, means that Guido was *conceptually*
on target with the "/" redefinition.)
Skip Montanaro
Greg> There should have been an operator in Python that says when
Greg> objects are conceptually the same. For the moment, call it "eq".
Greg> The following would hold:
Greg> 3 eq 3
Greg> not (3 eq 4)
Greg> a eq a # PROVIDED a HAS A VALUE
Greg> "foo" eq "foo"
Greg> not ([] eq [])
Greg> not ({} eq {})
Greg> () eq ()
Guido's time machine strikes again:
>>> 3 is 3
True
>>> not (3 is 4)
True
>>> a = "sdfsdf"
>>> a is a
True
>>> "foo" is "foo"
True
>>> not ([] is [])
True
>>> not ({} is {})
True
>>> () is ()
True
--
Skip Montanaro (sk...@pobox.com - http://www.mojam.com/)
Brian Quinlan
> Greg> 3 eq 3
> Greg> not (3 eq 4)
> Greg> a eq a # PROVIDED a HAS A VALUE
> Greg> "foo" eq "foo"
> Greg> not ([] eq [])
> Greg> not ({} eq {})
> Greg> () eq ()
>
> Guido's time machine strikes again:
>
> >>> 3 is 3
> True
This is an implementation accident.
>>> a = 100000
>>> b = 100000
>>> a is b
0
I think that Greg's proposed operator would return true in this case.
Cheers,
Brian
Aahz
As Greg pointed out, the first one and the last four are implementation
dependent. Do *NOT* rely on them.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
What if there were no rhetorical questions?
Philip Swartzleonard
> There should have been an operator in Python that says when objects
> are conceptually the same. For the moment, call it "eq". The
> following would hold:
>
> 3 eq 3
> not (3 eq 4)
> a eq a # PROVIDED a HAS A VALUE
> [etc, etc]
Well, if a is defined, a has a value. So this is either true or a
NameError (?). Or do you mean if a is None, a eq a should be false?
--
Philip Sw "Starweaver" [rasx] :: www.rubydragon.com
Fernando Pérez
Alex Martelli wrote:
> Your intuition, which led you astray here (Python does just
> what it should do), can be trained in several ways. The
> works of J. L. Borges and I. Calvino, if you like fiction
> that's reasonably sophisticated but still quite pleasant,
> are good bets. If you like non-fiction written by
> engineers fighting hard to dispell some of the errors of
> philosophers, Wittgenstein and Korzibsky are excellent.
>
> I'm not kidding, but I realize that many Pythonistas don't
> really care for either genre. In which case, this group
> and its archives, essays by GvR and /F, and the Python
> sources, may also prove interesting reading.
>
Well, I certainly hope your post encourages some to get acquainted with two of
the most elegant and fantastic writers of the 20th century. Unfortunately I
don't read Italian so I need to read Calvino translated, but there are few
joys in life which compare to reading Borges in his native Spanish
(especially his poetry; by the way, I think much of his work is by now
available in English).
And I would argue that the writings from these two gentlemen _should_ appeal
to the logically oriented mind of any programmer worth his salt.
You made my day with a reference here to some of my deepest 'intellectual
loves' :)
Regards,
f.
Petr Prikryl
>
> [...]
> There should have been an operator in Python that
> says when objects are conceptually the same.
The problem here is what does it mean
*conceptually*. For me *conceptually* means that
type([]) == type([]), but also
type([]) == type([1, 2, 3]). In other words,
the *list* as the abstraction is the concept.
> For the moment, call it "eq". The following would
> hold:
>
> 3 eq 3
> not (3 eq 4)
> a eq a # PROVIDED a HAS A VALUE
> "foo" eq "foo"
> not ([] eq [])
> not ({} eq {})
> () eq ()
>
> [...]
>
> Python almost has an "eq" operator. "==" is like
> "eq" with two exceptions:
>
> lists
> dictionaries
>
> Since "==" is so close to "eq", it is natural to
> think of "==" as meaning conceptual sameness.
> And if "==" meant conceptual sameness, then lists
> would be immutable, similar to Lisp lists (in the
> absence of rplaca and rplacd). In that case,
> .append(3) would have been implemented by
> consing, and your intuition would have been
> right.
For me, nor ``==`` nor ``is`` means *conceptual
sameness*, as said on the top of this message.
There are two distinct things when comparing the
objects: the object content and the object
identity. When comparing two objects' content,
the content is the same if the structure and
the related element values inside the compared
objects are equal. In other words, there is no
need for checking the objects identity in such
case; however, when the objects are identical
(i.e. one object) then the comparison can be
optimized to early return of true value.
Whith respect to this the ``==`` should compare
the content of comparable objects and the ``is``
operator should compare whether the identity of
the objects is equal. For me, Python does
exactly this.
> PS: Interestingly enough, all the expressions
> involving "eq" above are true on my system with
> "eq" replaced by "is". But (I believe) the
> following are implementation-dependent:
>
> 3 eq 3
> "foo" eq "foo"
> () eq ()
>
> And "eq" disagrees with "is" for (properly
> defined) immutable class instances. And,
> although I didn't mention it above, 3 eq 3.0
> should probably hold. (Which, I should add,
> means that Guido was *conceptually* on target
> with the "/" redefinition.)
The Python Reference Manual (Release 2.2.1),
section *5.9 Comparisons* says:
The operators is and is not test for object
identity: x is y is true if and only if x and
y are the same object. x is not y yields the
inverse truth value.
From this point of view, the ``3 is 3.0`` should
never hold because they are of different types
(they never can be the same object); but it
does not make sense to ask so either -- see below.
The ``3 == 3.0`` holds, because there is one more
step of the implicit conversion from int to float
and comparison of two float values (correct me,
if I am wrong).
The ``[]``, ``{}``, and ``()`` are in fact
constructors of new objects of the related type,
while ``3`` or ``"foo"`` are literals that may or
may not produce new object -- see below.
So, the ``[] is []`` should never hold, because
we are creating two instances of the list
abstraction. On the other hand, the ``[] == []``
should always hold, because the content of the
two empty lists is equal.
> Furthermore, two "eq" objects should hash to the
> same value. So *any* object could be a hash key.
It would be easier to implement this if there was
no keys() method of the dictionary, i.e. when you
used your *any object* content only to generate
the hash value and the hash value was used as the
key. But then, there would be no back way from
the hash to the key value. Moreover, two different
keys may produce the same hash values and the
dictionary structure has to discover that kind of
conflict and to do some more steps to make the
different keys really different.
If you think more about it, you will find that
Python requirements are natural, the key must be
immutable and even something more... The
immutable key (like a tuple) must not contain
references to mutable objects, because the
references are automatically replaced by the
value of the object when the hash value is to be
determined (correct me if I am wrong).
Brian Quinlan wrote in the answer iside this thread ...
>
> > >>> 3 is 3
> > True
>
> This is an implementation accident.
... On the other hand, what meaningful result one
should expect from ``3 is 3``? For me, ``3 == 3``
makes sense, but to ask whether ``3 is 3``
(i.e. whether the objects are identical) is
questionable. The Python Reference Manual
(Release 2.2.1) explains:
5.2.2 Literals
[... string, numbers, ...]
All literals correspond to immutable data
types, and hence the object's identity is less
important than its value. Multiple evaluations
of literals with the same value (either the
same occurrence in the program text or a
different occurrence) may obtain the same
object or a different object with the same
value.
See you,
Petr
--
Petr Prikryl (prikrylp at skil dot cz)
Michael Hudson
OK, bits of this thread are now 22k of xhtml1-strict-y goodness at:
http://starship.python.net/crew/mwh/hacks/objectthink.html
Please read and tell me about typos, misattributions and any other
sillynesses.
Cheers,
M.
--
I'm about to search Google for contract assassins to go to Iomega
and HP's programming groups and kill everyone there with some kind
of electrically charged rusty barbed thing.
-- http://bofhcam.org/journal/journal.html, 2002-01-08
Greg Weeks
From: "Petr Prikryl" <Answer.via.news...@skil.nospam.cz>
> There are two distinct things when comparing the objects: the object
> content and the object identity.
Neither of these concepts corresponds to conceptual sameness. I haven't
defined "conceptual sameness", but it is a natural concept:
Suppose a Bank_account type has the single attribute "balance". Consider
my_account = Bank_account(50) # INITIAL BALANCE OF $50
my_account.deposit(50) # BALANCE IS NOW $100
your_account = Bank_account(100) # INITIAL BALANCE OF $100
Do we have the same bank account? Of course not. Adding money to your
bank account is different from adding money to my bank account. So, in
this case, conceptual sameness corresponds to sameness of identity. The
same goes for lists and dictionaries, for the same reasons.
On the other hand, suppose we define our own Complex number class.
x = Complex(3,4)
y = Complex(3,0) + Complex(0,4)
Are x and y the same? Of course. There is only one 3+i4. In this case,
conceptual sameness corresponds to sameness of content.
A final example is immutable strings, which we imagine to be implemented as
a character array plus start and stop indices. (This implementation
permits a rapid substring operation. Note that the three attributes are
not allowed to be modified after they are initialized.) Consider:
s1 = String(['c', 'a', 't'], 1, 3)
s2 = String(['a', 't', 'e'], 0, 2)
Are these the same string? Conceptually, yes. When you are comparing
strings in your code, it will be conceptual sameness that you are
interested in. But neither the identity nor the content of s1 and s2 are
the same.
So, conceptual sameness corresponds sometimes to sameness of identity,
sometimes to sameness of content, and sometimes to neither. And (I
maintain), in ordinary user code, conceptual sameness is the concept of
*identity* that is most useful. (I stress "identity" because there are
weaker forms of comparison that are sometimes useful. But not *as*
useful.)
So Python is missing THE single most useful comparision operator. But
there are workarounds. "==" means conceptual sameness for all objects
other than lists and dictionaries, provided that __cmp__ is appropriately
defined. Furthermore, if __hash__ is appropriately defined, any object
other than a list or dictionary can be a hash key. So, except for lists
and dictionaries, everything's fine.
From: Philip Swartzleonard <st...@pacbell.net>
> > a eq a # PROVIDED a HAS A VALUE
>
> Well, if a is defined, a has a value. So this is either true or a
> NameError (?). Or do you mean if a is None, a eq a should be false?
No. I was perhaps overfastidiously ruling out:
>>> a is a
Traceback (innermost last):
File "<stdin>", line 1, in ?
NameError: a
Regards,
Greg
Steve Holden
assert...
> Some comments regarding "conceptual sameness":
>
>
> From: "Petr Prikryl" <Answer.via.news...@skil.nospam.cz>
>
> > There are two distinct things when comparing the objects: the object
> > content and the object identity.
>
> Neither of these concepts corresponds to conceptual sameness. I haven't
> defined "conceptual sameness", but it is a natural concept:
>
you can do it.
[spurious examples]
>
> Are these the same string? Conceptually, yes. When you are comparing
> strings in your code, it will be conceptual sameness that you are
> interested in. But neither the identity nor the content of s1 and s2 are
> the same.
>
> So, conceptual sameness corresponds sometimes to sameness of identity,
> sometimes to sameness of content, and sometimes to neither. And (I
> maintain), in ordinary user code, conceptual sameness is the concept of
> *identity* that is most useful. (I stress "identity" because there are
> weaker forms of comparison that are sometimes useful. But not *as*
> useful.)
>
Well, if you wave your hands long enough you might persuade some readers
that this all *means* something. But I'm not convinced you are yet at the
stage where you could provide a patch to add a "conceptual sameness"
operator to the language. Clearly object identity and object equality
*might* be equivalent for some types/classes.
At present Python, with its typically pragmatic approach to these matters,
does not insist on single-copy instances for immutable types. Consequently
it is possible for a == b to be true even though a is not b.
> So Python is missing THE single most useful comparision operator. But
> there are workarounds. "==" means conceptual sameness for all objects
> other than lists and dictionaries, provided that __cmp__ is appropriately
> defined. Furthermore, if __hash__ is appropriately defined, any objec
> other than a list or dictionary can be a hash key. So, except for lists
> and dictionaries, everything's fine.
>
I don't feel you have yet even adequately *defined* "conceptual sameness",
so it's surely premature to assert that it's missing from the language.
regards
Steve
--
home: http://www.holdenweb.com/ book: http://pydish.holdenweb.com/pwp/
Tim Roberts
>Some comments regarding "conceptual sameness":
>
>On the other hand, suppose we define our own Complex number class.
>
> x = Complex(3,4)
> y = Complex(3,0) + Complex(0,4)
>
>Are x and y the same? Of course. There is only one 3+i4. In this case,
>conceptual sameness corresponds to sameness of content.
I disagree. I don't think you can say "there is only one 3+i4" any more
than you can say "there is only one string 'abc'". We're deeply into
semantics and meaning here, but correct semantics are critically important
to programming.
I would say that "x and y are equal", but I would not say that "x and y are
the same". For example, given this next:
x += Complex(1,0)
If x and y are the same, I would expect this to alter y.
--
- Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
Dave Kuhlman
also look at the 'is' operator and the built-in function id().
a is b if and only if id(a) == id(b)
The id() function will tell you whether you are dealing with a
reference to the same object or with a copy of the object.
domin...@yahoo.com wrote:
>
> Hello,
>
> one thing I like very much about Python is that statements
> work like you would expect them to work. Take for example
> the use of dict.values() for dictionaries: If you store the
> result of dict.values(), and change the dictionary after-
> wards, the previously stored result remains untouched.
[snip]
--
Dave Kuhlman
dkuh...@rexx.com
http://www.rexx.com/~dkuhlman
Greg Ewing
>
> I don't feel you have yet even adequately *defined* "conceptual sameness",
> so it's surely premature to assert that it's missing from the language.
The definition he seems to have in mind is something like
def eq(a, b):
if is_immutable(a) and is_immutable(b):
return a == b
else:
return a is b
for some suitable definition of is_immutable.
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg