Reference or Value?
Torsten Mohr
how is the rule in Python, if i pass objects to a function, when is this
done by reference and when is it by value?
def f1(a):
a = 7
b = 3
f1(b)
print b
=> 3
Integers are obviously passed by value, lists and dicts by reference.
Is there a general rule? Some common formulation?
Thanks for any hints,
Torsten.
Christian Heimes
Python uses neither call by reference nor call by value. It's called
call by sharing or call by object. http://effbot.org/zone/call-by-object.htm
Christian
Duncan Booth
> how is the rule in Python, if i pass objects to a function, when is this
> done by reference and when is it by value?
>
> def f1(a):
> a = 7
>
> b = 3
> f1(b)
> print b
>=> 3
>
> Integers are obviously passed by value, lists and dicts by reference.
>
> Is there a general rule? Some common formulation?
Yes, integers are 'obviously' passed by value, except that if they were you
couldn't do this:
>>> x = 26*1000
>>> y = 26000
>>> id(x)==id(y)
False
>>> def whichint(arg):
if id(arg)==id(x):
print "You passed x"
elif id(arg)==id(y):
print "You passed y"
>>> whichint(x)
You passed x
>>> whichint(y)
You passed y
The general rule is that everything is passed in exactly the same way.
MRAB
def f2(a):
a = [7]
b = [3]
f2(b)
print b
=> [3]
It's just that lists and dicts are containers whose contents you can
change (they are mutable), but integers aren't containers (they are
immutable).
Tuples are also containers, but you can't change their contents (they
are immutable).
andrew cooke
as far as i understand things, the best model is:
1 - everything is an object
2 - everything is passed by reference
3 - some objects are immutable
4 - some (immutable?) objects are cached/reused by the system
andrew
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
Andrew Koenig
news:mailman.464.12353206...@python.org...
> as far as i understand things, the best model is:
>
> 1 - everything is an object
> 2 - everything is passed by reference
> 3 - some objects are immutable
> 4 - some (immutable?) objects are cached/reused by the system
0 - Assignment rebinds the reference on its left-hand side; it does not
change the object to which that reference refers.
Example:
x = 42
y = x
Now x and y are bound to the same object
x = x + 1
This statement computes the value of x + 1, which is a new object with value
43. It then rebinds x to refer to this object, so x and y now refer to
different objects. Therefore:
def f(a):
a = a + 1
x = 42
f(x)
This example behaves analogously to the previous one: The assignment a = a +
1 binds a to a new object, so it does not affect the object to which x is
bound.
z = [3]
y = z
z[0] = z[0] + 1
The assignment rebinds z[0] to refer to a new object that has the value 4.
This rebinding does not affect the object formerly bound to z[0]. It does,
however, affect the value of the object to which z is bound, because it
changes the value of its list element. By analogy:
def g(b):
b[0] = b[0] + 1
w = [42]
g(w)
Now w[0] will be 43.
Steven D'Aprano
> Hi,
>
> how is the rule in Python, if i pass objects to a function, when is this
> done by reference and when is it by value?
Never, and never.
> Integers are obviously passed by value, lists and dicts by reference.
Your error is assuming that pass-by-value and pass-by-reference are the
only two possibilities, consequently you are misinterpreting what you see.
Python uses the same evaluation strategy regardless of the type of object
being passed. Sometimes that strategy looks something like pass-by-value,
and sometimes it looks something like pass-by-reference, but in fact it
always behaves consistently no matter what object you use.
> Is there a general rule? Some common formulation?
http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing
http://effbot.org/zone/call-by-object.htm
--
Steven
Steven D'Aprano
> as far as i understand things, the best model is:
>
> 1 - everything is an object
> 2 - everything is passed by reference
Except that is wrong. If it were true, you could do this:
def swap(x, y):
y, x = x, y
a = 1
b = 2
swap(a, b)
assert a == 2 and b == 1
but you can't, it does not work. Ergo, parameter passing in Python does
not have the same semantics as languages that use pass-by-reference, such
as Pascal and Basic. That means that even if you can justify the claim
"Python is pass-by-reference" by some technical argument (and I don't
believe you can), it is misleading to make that claim without further
qualifications.
> 3 - some objects are immutable
All objects are passed the same way, regardless of whether they are
mutable or immutable.
> 4 - some (immutable?) objects are cached/reused by the system
That's irrelevant. Caching affects the creation of new objects, but has
no effect on argument passing.
--
Steven
Denis Kasak
<ste...@remove.this.cybersource.com.au> wrote:
> On Sun, 22 Feb 2009 13:37:27 -0300, andrew cooke wrote:
>
>> as far as i understand things, the best model is:
>>
>> 1 - everything is an object
>> 2 - everything is passed by reference
>
> Except that is wrong. If it were true, you could do this:
>
> def swap(x, y):
> y, x = x, y
>
> a = 1
> b = 2
> swap(a, b)
> assert a == 2 and b == 1
>
>
> but you can't, it does not work. Ergo, parameter passing in Python does
> not have the same semantics as languages that use pass-by-reference, such
> as Pascal and Basic. That means that even if you can justify the claim
> "Python is pass-by-reference" by some technical argument (and I don't
> believe you can), it is misleading to make that claim without further
> qualifications.
You could, however, argue that the swap function doesn't work as
expected (e.g. from a Pascal or a C++ POV) simply because the
underlying objects aren't mutable. The objects *do* get passed by
reference; the function doesn't receive a new copy of the object and
it can examine the original object's ID. The actual culprit is not the
way objects are passed but the assignment operator, since it works by
rebinding names (as Andrew Koenig explained) and not by changing the
object itself. If the swap() function could somehow access the
underlying integer object and modify it, swapping of values would
indeed occur because the function *did* get references to the objects
passed to it.
That said, it's a rather convoluted way of explaining what happens and
calling it pass-by-object feels much better. :-)
--
Denis Kasak
afr...@yahoo.co.uk
Let's add some comments and a little code
>>>def f1 (a) :
print a, id(a)
#print the object passed and it's id
a = 7
#reassign 'a' to point to a different object
print a, id(a)
#print the new object and it's id
>>> b = 1
>>> #make 'b' refer to object '1' residing at a particular memory location
>>> id(b)
161120104
>>> #which we might refer to as 161120104
>>> f1(b) #pass the object '1' refered to by 'b' to our function
1 161120104
7 161120032
>>> #no surprises there, the new object has a different id.
>>> id(b)
161120104
>>> #no surprises ther, b (unlike the name internal to the function
>>> #hasn't been reassigned, so it still points where it always has
>>> #which is of course ..
>>> b
1
>>> z = []
>>> #name 'z' refers to an empty list
>>> id(z)
162432780
>>> f1(z)
[] 162432780
7 161120032
>>> #no surprise that the empty list object has the same id outside
>>> #and inside the function
>>> #stranger is that the '7' object has the same id as the 7 object
>>> #created on the previous run on the function, but that is another
>>> #story
Gabriel Genellina
escribió:
> On Mon, Feb 23, 2009 at 5:09 AM, Steven D'Aprano
> <ste...@remove.this.cybersource.com.au> wrote:
>> On Sun, 22 Feb 2009 13:37:27 -0300, andrew cooke wrote:
>>
>>> as far as i understand things, the best model is:
>>>
>>> 1 - everything is an object
>>> 2 - everything is passed by reference
>>
>> Except that is wrong. If it were true, you could do this:
>>
>> def swap(x, y):
>> y, x = x, y
>>
>> a = 1
>> b = 2
>> swap(a, b)
>> assert a == 2 and b == 1
>>
>>
>> but you can't, it does not work. Ergo, parameter passing in Python does
>> not have the same semantics as languages that use pass-by-reference,
>> such
>> as Pascal and Basic. That means that even if you can justify the claim
>> "Python is pass-by-reference" by some technical argument (and I don't
>> believe you can), it is misleading to make that claim without further
>> qualifications.
>
> You could, however, argue that the swap function doesn't work as
> expected (e.g. from a Pascal or a C++ POV) simply because the
> underlying objects aren't mutable.
That's irrelevant - mutable and immutable objects are passed exactly the
same way.
> The objects *do* get passed by
> reference; the function doesn't receive a new copy of the object and
> it can examine the original object's ID. The actual culprit is not the
> way objects are passed but the assignment operator, since it works by
> rebinding names (as Andrew Koenig explained) and not by changing the
> object itself.
There is *no* difference between "the way objects are passed" and "the
assignment opera[tion]": both work exactly the same way.
Python execution model is based on namespaces (a collection of
name->object pairs, usually implemented as dictionaries). An assignment x
= y means:
- evaluate the right hand side and obtain a value (an object, always). In
this case, it is the object referenced by the name "y"
- make the name "x" refer to such object. That's all.
In short, assignment rebinds a name in a namespace.
When a function call is made, a new namespace is created. The names come
from the function parameters (the names that were used to define the
function in the def ... line) and the objects come from the actual
arguments (the objects you pass inside the (...) when doing the call). The
function code is then executed using this namespace as its locals().
def foo(a, b):
print a, b
foo(4, ['hello', 'world!'])
The call is equivalent to:
ns = {}
ns['a'], ns['b'] = 4, ['hello', 'world!']
execute foo code using ns as locals
except the right hand side is evaluated on the *calling* namespace, and
the left hand assignments are done on the new namespace. So a call binds
many names in a new namespace: the same as assignment.
No copies, no pointers, nothing: only names and objects are manipulated.
> If the swap() function could somehow access the
> underlying integer object and modify it, swapping of values would
> indeed occur because the function *did* get references to the objects
> passed to it.
There is a difference between modify the value of an object, and modify
the caller's namespace. Usually "pass by reference" means that the
*caller* may see a different thing after the call - this is not true in
Python, there is no way the called code could alter the caller namespace
(well, not just due to the call operation alone...)
> That said, it's a rather convoluted way of explaining what happens and
> calling it pass-by-object feels much better. :-)
And is more correct...
--
Gabriel Genellina
andrew cooke
> On Sun, 22 Feb 2009 13:37:27 -0300, andrew cooke wrote:
>
>> as far as i understand things, the best model is:
>>
>> 1 - everything is an object
>> 2 - everything is passed by reference
>
> Except that is wrong. If it were true, you could do this:
i was thinking of how the stack is used; i would have called what you are
talking about "pointer semantics". however, on reading around a little,
it seems that i'm in a very small minority (which is a pity, because if
you restrict the meaning to how values are handled on the stack then you
get a lot closer to having just values and references, rather than the
whole pile of different terms that are apparently in use).
sorry for the confusion,
andrew
Steve Holden
> On Mon, Feb 23, 2009 at 5:09 AM, Steven D'Aprano
> <ste...@remove.this.cybersource.com.au> wrote:
>>
>>> as far as i understand things, the best model is:
>>>
>>> 1 - everything is an object
>>> 2 - everything is passed by reference
>> Except that is wrong. If it were true, you could do this:
>>
>> def swap(x, y):
>> y, x = x, y
>>
>> a = 1
>> b = 2
>> swap(a, b)
>> assert a == 2 and b == 1
>>
>>
>> but you can't, it does not work. Ergo, parameter passing in Python does
>> not have the same semantics as languages that use pass-by-reference, such
>> as Pascal and Basic. That means that even if you can justify the claim
>> "Python is pass-by-reference" by some technical argument (and I don't
>> believe you can), it is misleading to make that claim without further
>> qualifications.
>
> expected (e.g. from a Pascal or a C++ POV) simply because the
> reference; the function doesn't receive a new copy of the object and
> it can examine the original object's ID. The actual culprit is not the
> way objects are passed but the assignment operator, since it works by
> rebinding names (as Andrew Koenig explained) and not by changing the
> underlying integer object and modify it, swapping of values would
> indeed occur because the function *did* get references to the objects
> passed to it.
>
> calling it pass-by-object feels much better. :-)
>
caller's namespace by manipulations of parameters passed as arguments to
a function.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Christian Heimes
> You could, however, argue that the swap function doesn't work as
> expected (e.g. from a Pascal or a C++ POV) simply because the
> underlying objects aren't mutable. The objects *do* get passed by
> reference;
We are getting down the same road every couple of months. Please don't
explain Python's calling system with terms like "call by reference".
It's plain simple wrong. The correct term is "call by sharing" or "call
by object reference" although I myself don't use the latter term because
it sounds too much like "call by reference". Every time somebody tries
to explain Python with "call by reference", the person is confusing
himself and others.
Christian
Denis Kasak
I assure you I am not confused about Python's object model / calling
system. I was arguing, from a purely theoretical standpoint, that the
same system Python uses could be described in terms of
call-by-reference with some additional constraints. I am by no means
arguing that this is a good way of explaining it or trying to explain
it to someone in terms of call-by-reference. I just don't think it's
"plain simple wrong", just confusing and suboptimal.
--
Denis Kasak
Steve Holden
Regulars on c.l.py see this subject arising so regularly it's a bit like
"there's nothing new under the sun". Honestly, it's been discussed /ad
nauseam/ lately.
Denis Kasak
> Denis Kasak wrote:
>> I assure you I am not confused about Python's object model / calling
>> system. I was arguing, from a purely theoretical standpoint, that the
>> same system Python uses could be described in terms of
>> call-by-reference with some additional constraints. I am by no means
>> arguing that this is a good way of explaining it or trying to explain
>> it to someone in terms of call-by-reference. I just don't think it's
>> "plain simple wrong", just confusing and suboptimal.
>>
> Well, what's the benefit of discussing such a description, then?
> Regulars on c.l.py see this subject arising so regularly it's a bit like
> "there's nothing new under the sun". Honestly, it's been discussed /ad
> nauseam/ lately.
Someone said upthread that Andrew Cooke was completely wrong and I
contested that. :-)
I apologise for not being current with the latest discussions.
--
Denis Kasak
Steve Holden
Gabriel Genellina
escribió:
> On Feb 23, 2009, at 3:03 , Gabriel Genellina wrote:
>> En Mon, 23 Feb 2009 03:54:16 -0200, Denis Kasak
>> <denis...@gmail.com> escribió:
>>> <ste...@remove.this.cybersource.com.au> wrote:
>>>>
>>>>> as far as i understand things, the best model is:
>>>>>
>>>>> 1 - everything is an object
>>>>> 2 - everything is passed by reference
>>>>
>>>> Except that is wrong. If it were true, you could do this:
>>>>
>>>> def swap(x, y):
>>>> y, x = x, y
>>>>
>>>> a = 1
>>>> b = 2
>>>> swap(a, b)
>>>> assert a == 2 and b == 1
>>>>
>>>>
>>>> but you can't, it does not work. Ergo, parameter passing in
>>>> Python does
>>>> not have the same semantics as languages that use pass-by-
>>>> reference, such
>>>> as Pascal and Basic.
>>>
>>> expected (e.g. from a Pascal or a C++ POV) simply because the
>>> underlying objects aren't mutable.
>>
>> exactly the same way.
>
> were handled differently. They are different in terms of what a
> function can do to them (mutate them or no). Thus, if you can't
> mutate them, then the caller can't see any changes in the objects by
> actions in the function so the swap won't work: it doesn't mutate the
> objects at all. In the case of a list or a dict, then one can mutate
> them, and the changes are seen in the caller. In both cases, the
> object itself is passed the same way.
Ok, in any case, making such claims without explanation or restricting the
sense, only confuses the matter (which is innecesarily confusing
already...)
--
Gabriel Genellina
Steven D'Aprano
Why is it a pity to have a whole pile of different terms to describe a
whole lot of different behaviours? I would suggest the real confusion
would be if we had two terms to describe a dozen different parameter-
passing conventions.
--
Steven
andrew cooke
because:
(1) it's often useful to explain things as (simpler) orthogonal components
rather than globbing everything under one term; that lets you more easily
carry across knowledge from previous experience.
(2) while using a term that has (according to wikipedia) "widespread usage
in the Python community" helps you save time and avoid confusion, it
doesn't necessarily explain best to someone from outside that community
what is happening. in this particular case you are getting dangerously
close to having a term that is used only for one language and which,
therefore, is at risk of meaning little more that "the way python does it"
(which is, i think you'll agree, more tautological than helpful).
andrew
Gabriel Genellina
escribió:
> because:
>
> (1) it's often useful to explain things as (simpler) orthogonal
> components
> rather than globbing everything under one term; that lets you more easily
> carry across knowledge from previous experience.
>
> (2) while using a term that has (according to wikipedia) "widespread
> usage
> in the Python community" helps you save time and avoid confusion, it
> doesn't necessarily explain best to someone from outside that community
> what is happening. in this particular case you are getting dangerously
> close to having a term that is used only for one language and which,
> therefore, is at risk of meaning little more that "the way python does
> it"
> (which is, i think you'll agree, more tautological than helpful).
But -paraphrasing A.E.- we should explain it in the simplest terms, but
not simpler.
Some people think that there exist only two possible ways to pass an
argument: by value or by reference. The way Python does it doesn't have
the same properties that pass-by-value nor pass-by-reference have in other
languages, so both names are inapropiate. You appear to favor the "by
reference" name. A couple months ago some other guy were strongly
defending the "by value" name -- he even set up a website supporting his
idea.
The fact is, if you say "it's by value" you'll disappoint some
expectations from some users, and if you say "it's by reference" you'll
disappoint some other expectations. So you must say something different
("it's by reference, but the references cannot be modified (!), and...",
"it's by value, but the values are references (!!), and...", or whatever
description you choose). Given that, it's better to use a different name:
you're going to explain the concept anyway.
And Python isn't the first language using this convention; if you look in
the FAQ or search recent threads in this group you'll find references
going back to the '70s.
--
Gabriel Genellina
andrew cooke
> En Tue, 24 Feb 2009 06:44:01 -0200, andrew cooke <and...@acooke.org>
> escribió:
>
if you read my post i think you'll find i was not advocating either of those.
i really think this discussion has gone on far enough; i have better
things to do.
andrew
> And Python isn't the first language using this convention; if you look in
> the FAQ or search recent threads in this group you'll find references
> going back to the '70s.
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
Steven D'Aprano
"One term"?
If you look at this Wikipedia page:
http://en.wikipedia.org/wiki/Evaluation_strategy
you should be able to count no fewer than thirteen different terms for
different evaluation strategies, and "call-by-value" itself is described as
a family of strategies. Who is gobbing everything under one term? Certainly
not me. If anything, your preferred tactic is to gob everything under one
of two terms, call-by-value and call-by-reference. That's not my preferred
tactic.
> that lets you more easily
> carry across knowledge from previous experience.
That's only useful if that previous experience *clarifies* rather than
*confuses*. This very thread demonstrates that people's prior knowledge of
call-by-whatever doesn't clarify Python's behaviour.
> (2) while using a term that has (according to wikipedia) "widespread usage
> in the Python community" helps you save time and avoid confusion, it
> doesn't necessarily explain best to someone from outside that community
> what is happening.
And you think call-by-reference does?
Python demonstrably does not behave like call-by-reference in other
languages. I do not agree that it is wise to hammer the round peg of
Python's actual behaviour into the square peg of newbies' concept of
call-by-reference.
> in this particular case you are getting dangerously
> close to having a term that is used only for one language
So what? What if Python was the only language that used the term? That
doesn't make it inappropriate. If the users of other languages that use
call-by-object semantics (Java, RealBasic, etc) choose to prefer a
misleading name over a descriptive name, that's no reason for Python users
to follow down their path.
> and which,
> therefore, is at risk of meaning little more that "the way python does it"
> (which is, i think you'll agree, more tautological than helpful).
No at all. Do you imagine that call-by-name means "the way Algol does it"?
Algol is the only major language that uses call-by-name, it may even be the
only language of any size, but that doesn't mean that call-by-name can't be
defined and described independently of Algol. There's even a test,
Knuth's "Man Or Boy Test", for testing how well a language implements
call-by-name semantics.
--
Steven
Tim Rowe
> If you look at this Wikipedia page:
>
> http://en.wikipedia.org/wiki/Evaluation_strategy
>
> you should be able to count no fewer than thirteen different terms for
> different evaluation strategies, and "call-by-value" itself is described as
> a family of strategies.
And that call by sharing is a member of that family even though the
behaviour is (as I understand it) exactly what you would get if you
pass an object by ^reference^ in C++. Wikipedia can be a double-edged
sword ;-).
I think the problem is that "Call by value" and "Call by reference"
are terms that pre-date object orientation, and don't cope well with
obect-oriented languages in general, not just with Python. That Java
programmers get by thinking the mechanism is called "call by value",
C++ programmers get by thinking the same mechanism is called "call by
reference" and Python programmers get by thinking the same mechanism
is called "call by sharing" suggests that the terms are no longer
helpful. Programmers in one language simply can't know what the
terminology means when applied to another language without learning
the behaviour in that other language before learning the terminology.
That means that Torsten's original question probably wasn't the one he
wanted to ask, and wasn't a useful one. The useful question is "what
is Python's parameter passing *behaviour^",
with a possible supplementary "what do they call that behaviour *in
the Python community*", recognising that if they asked "what is Java's
parameter passing *behaviour^",
with the supplementary "what do they call that behaviour *in the Java
community*" they would get much the same answer for the first question
(immutability is handled differently, I think, but it's there) but a
totally different answer to the second.
--
Tim Rowe