Help understanding the decisions *behind* python?
Phillip B Oldham
now, and while we love a lot of what python has done for us and what
it enables us to do some of the decisions behind such certain
data-types and their related methods baffle us slightly (when compared
to the decisions made in other, similarly powerful languages).
Specifically the "differences" between lists and tuples have us
confused and have caused many "discussions" in the office. We
understand that lists are mutable and tuples are not, but we're a
little lost as to why the two were kept separate from the start. They
both perform a very similar job as far as we can tell.
Consider the following:
>>> x = [2,1,3]
>>> x.sort()
>>> print x
[1, 2, 3]
Now, if the sort operations were unable to affect the original
structure of the list (as in JavaScript) you'd effectively have a
tuple which you could add/remove from, and the example above would
look more like:
>>> x = [2,1,3]
>>> print x.sort()
[1, 2, 3]
>>> print x
[2,1,3]
This make a lot more sense to us, and follows the convention from
other languages. It would also mean chaining methods to manipulate
lists would be easier:
>>> x = [2,1,3]
>>> print x.sort()[0]
3
>>> print x
[2,1,3]
We often find we need to do manipulations like the above without
changing the order of the original list, and languages like JS allow
this. We can't work out how to do this in python though, other than
duplicating the list, sorting, reversing, then discarding.
We're not looking to start any arguments or religious wars and we're
not asking that python be changed into something its not. We'd simply
like to understand the decision behind the lists and tuple structures.
We feel that in not "getting" the difference between the two types we
may be missing out on using these data structures to their full
potential.
Tycho Andersen
Oldham<phillip...@gmail.com> wrote:
> <snip>
> We often find we need to do manipulations like the above without
> changing the order of the original list, and languages like JS allow
> this. We can't work out how to do this in python though, other than
> duplicating the list, sorting, reversing, then discarding.
>
I have no idea about why the design decisions were made. You might
take a look at the sorted() function:
http://docs.python.org/library/functions.html#sorted
It will do what you want.
\t
--
http://tycho.ws
Duncan Booth
> This make a lot more sense to us, and follows the convention from
> other languages. It would also mean chaining methods to manipulate
> lists would be easier:
>
>>>> x = [2,1,3]
>>>> print x.sort()[0]
> 3
>>>> print x
> [2,1,3]
You already have a way to do what you want:
>>> x = [2,1,3]
>>> print sorted(x)[0]
3
>>> print x
[2,1,3]
as a bonus you can use 'sorted' to sort any sequence including generators
or tuples, but the result will always be a list: if it was a list method
then you would have to convert the sequence to a list first.
The main reason why you need both lists and tuples is that because a tuple
of immutable objects is itself immutable you can use it as a dictionary
key. You can't use a list as a dictionary key because if something were to
mutate a key the dictionary structure would behave very strangely indeed.
The types 'set' and 'frozenset' both exist for the same reason.
Michiel Overtoom
> Specifically the "differences" between lists and tuples have us
> confused and have caused many "discussions" in the office. We
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
Yes, but because of their immutability you can use tuples as dictionary
keys (only if they contain immutable objects).
Lists can't be used as dictionary keys.
Greetings,
--
"The ability of the OSS process to collect and harness
the collective IQ of thousands of individuals across
the Internet is simply amazing." - Vinod Valloppillil
http://www.catb.org/~esr/halloween/halloween4.html
Anthony Tolle
wrote:
> ...
> Specifically the "differences" between lists and tuples have us
> confused and have caused many "discussions" in the office. We
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
There's no hard-set rules, but tuples are typically used to hold
collections of heterogeneous data, e.g. (10, "spam", 3.21). As has
been mentioned, such a tuple can be used as a dictionary key, whereas
a list cannot be used as a dictionary key, because it is mutable.
Lists, on the other hand, typically hold collections of homogeneous
data, e.g. [1, 2, 5] or ["spam", "eggs", "sausage"].
Of course, you'll also see plenty of examples of tuples containing
homogeneous data and lists containing heterogeneous data :)
Phillip B Oldham
> The main reason why you need both lists and tuples is that because a tuple
> of immutable objects is itself immutable you can use it as a dictionary
> key.
Really? That sounds interesting, although I can't think of any real-
world cases where you'd use something like that.
Duncan Booth
How about a multi-dimensional sparse array?
>>> d = {}
>>> d[1,2] = 'a'
>>> d[5,7] = 'b'
Marcus Wanner
to synchronize typed text and speech ?"...i.e. have a dictionary key of
(hour, minute, second).
Marcus
J. Cliff Dyer
Well, if you wanted to index a dictionary by coordinates, you might do
something like this:
fleet = {}
fleet[9,4] = 'destroyer'
fleet[8,4] = 'destroyer'
fleet[3,5] = 'aircraftcarrier'
fleet[4,5] = 'aircraftcarrier'
fleet[5,5] = 'aircraftcarrier'
fleet[6,5] = 'aircraftcarrier'
fleet[8,0] = 'battleship'
fleet[8,1] = 'battleship'
fleet[8,2] = 'battleship'
def checkattack(x, y, fleet):
if x,y in fleet:
return "You hit my %s' % fleet[x,y]
Maybe not the best implementation of Battleship, but you get the idea.
Piet van Oostrum
>DB> Phillip B Oldham <phillip...@gmail.com> wrote:
>>> This make a lot more sense to us, and follows the convention from
>>> other languages. It would also mean chaining methods to manipulate
>>> lists would be easier:
>>>
>>>>>> x = [2,1,3]
>>>>>> print x.sort()[0]
>>> 3
>>>>>> print x
>>> [2,1,3]
>DB> You already have a way to do what you want:
>>>>> x = [2,1,3]
>>>>> print sorted(x)[0]
>DB> 3
What kind of Python produces that?
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: pi...@vanoostrum.org
Chris Rebert
>>>>>> Duncan Booth <duncan...@invalid.invalid> (DB) wrote:
>
>>DB> Phillip B Oldham <phillip...@gmail.com> wrote:
>>>> This make a lot more sense to us, and follows the convention from
>>>> other languages. It would also mean chaining methods to manipulate
>>>> lists would be easier:
>>>>
>>>>>>> x = [2,1,3]
>>>>>>> print x.sort()[0]
>>>> 3
>>>>>>> print x
>>>> [2,1,3]
>
>>DB> You already have a way to do what you want:
>
>>>>>> x = [2,1,3]
>>>>>> print sorted(x)[0]
>>DB> 3
>
> What kind of Python produces that?
Assuming you're referring to the latter example, it was added in version 2.4
If you meant the former example, I think that's purely pseudo-Python.
Cheers,
Chris
--
http://blog.rebertia.com
Paul Moore
>>>>>>> print sorted(x)[0]
>>>DB> 3
>>
>> What kind of Python produces that?
>
> If you meant the former example, I think that's purely pseudo-Python.
I think he was referring to the fact that the result should be 1, not 3.
Paul.
Niels L. Ellegaard
> We often find we need to do manipulations like the above without
> changing the order of the original list, and languages like JS allow
> this. We can't work out how to do this in python though, other than
> duplicating the list, sorting, reversing, then discarding.
If you just want a one-liner, and you don't care about speed you can
do the following (but I don't think this is considered best practice)
>>> x = [2,1,3]
>>> print list(sorted(x))
[1, 2, 3]
>>> print x
[2, 1, 3]
Niels
Christian Heimes
> Phillip B Oldham <phillip...@gmail.com> writes:
>
>> We often find we need to do manipulations like the above without
>> changing the order of the original list, and languages like JS allow
>> this. We can't work out how to do this in python though, other than
>> duplicating the list, sorting, reversing, then discarding.
>
> If you just want a one-liner, and you don't care about speed you can
> do the following (but I don't think this is considered best practice)
sorted() already returns a new list not a generator as you might assume.
Christian
Hrvoje Niksic
An application visiting files on a filesystem recursively needs a
dictionary or set keyed by (st_dev, st_ino) to make sure it doesn't
visit the same file twice. Memoization implementation (of a cache for
results of function application) might need to use a dictionary to map
function arguments, a tuple, to the function result.
Hrvoje Niksic
>>>>>>> x = [2,1,3]
>>>>>>> print sorted(x)[0]
>>>DB> 3
>>
>> What kind of Python produces that?
>
> Assuming you're referring to the latter example, it was added in version 2.4
> If you meant the former example, I think that's purely pseudo-Python.
sorted([2, 1, 3])[0] evaluates to 1, not 3.
Piet van Oostrum
>PM> 2009/7/20 Chris Rebert <cl...@rebertia.com>:
>>> On Mon, Jul 20, 2009 at 2:23 PM, Piet van Oostrum<pi...@cs.uu.nl> wrote:
>>>>>>>>> x = [2,1,3]
>>>>>>>>> print sorted(x)[0]
>DB> 3
>>>>
>>>> What kind of Python produces that?
>>>
>>> Assuming you're referring to the latter example, it was added in version 2.4
>>> If you meant the former example, I think that's purely pseudo-Python.
>PM> I think he was referring to the fact that the result should be 1, not 3.
The underlying thought was that you should copy and paste examples from
a real Python interpreter.
Hendrik van Rooyen
simplest is something like a point in the cartesian plane with
an associated attribute like colour.
- Hendrik
Duncan Booth
I guess you can tell I broke my unbreakable rule to never post untested
code without labelling it as such. :-)
--
Duncan Booth http://kupuguy.blogspot.com
Inky 788
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
My guess is that it was probably for optimization reasons long ago.
I've never heard a *good* reason why Python needs both.
Piet van Oostrum
>HvR> On Monday 20 July 2009 21:26:07 Phillip B Oldham wrote:
>>> On Jul 20, 6:08�pm, Duncan Booth <duncan.bo...@invalid.invalid> wrote:
>>> > The main reason why you need both lists and tuples is that because a
>>> > tuple of immutable objects is itself immutable you can use it as a
>>> > dictionary key.
>>>
>>> Really? That sounds interesting, although I can't think of any real-
>>> world cases where you'd use something like that.
>HvR> simplest is something like a point in the cartesian plane with
>HvR> an associated attribute like colour.
There are numerous other examples. Anytime you need a key that is not a
single object but composed of more than one:
Name + birthday
Street + number + City
Student + Course
etc.
David Smith
>>>>>> Hendrik van Rooyen <hen...@microcorp.co.za> (HvR) wrote:
>
>> HvR> On Monday 20 July 2009 21:26:07 Phillip B Oldham wrote:
>>>> On Jul 20, 6:08 pm, Duncan Booth <duncan.bo...@invalid.invalid> wrote:
>>>>> The main reason why you need both lists and tuples is that because a
>>>>> tuple of immutable objects is itself immutable you can use it as a
>>>>> dictionary key.
>>>> Really? That sounds interesting, although I can't think of any real-
>>>> world cases where you'd use something like that.
>
>> HvR> simplest is something like a point in the cartesian plane with
>> HvR> an associated attribute like colour.
>
> There are numerous other examples. Anytime you need a key that is not a
> single object but composed of more than one:
> Name + birthday
> Street + number + City
> Student + Course
> etc.
Compound keys (like what's listed above) can also be used for sorting
lists of dictionaries using DSU style sorting. Something I believe (and
I could be wrong) won't work with mutable types like lists.
--David
Piet van Oostrum
>DS> Piet van Oostrum wrote:
>>>>>>>> Hendrik van Rooyen <hen...@microcorp.co.za> (HvR) wrote:
>>>
>HvR> On Monday 20 July 2009 21:26:07 Phillip B Oldham wrote:
>>>>>> On Jul 20, 6:08 pm, Duncan Booth <duncan.bo...@invalid.invalid> wrote:
>>>>> The main reason why you need both lists and tuples is that because a
>>>>> tuple of immutable objects is itself immutable you can use it as a
>>>>> dictionary key.
>>>>>> Really? That sounds interesting, although I can't think of any real-
>>>>>> world cases where you'd use something like that.
>>>
>HvR> simplest is something like a point in the cartesian plane with
>HvR> an associated attribute like colour.
>>>
>>> There are numerous other examples. Anytime you need a key that is not a
>>> single object but composed of more than one:
>>> Name + birthday
>>> Street + number + City
>>> Student + Course
>>> etc.
>DS> Compound keys (like what's listed above) can also be used for sorting
>DS> lists of dictionaries using DSU style sorting. Something I believe (and
>DS> I could be wrong) won't work with mutable types like lists.
For sorting there is no problem with mutable arrays as long as you don't
mutate them during the sorting process (for example in the comparison
routine). Doing that would be extremely perverse. And there is no
enforcement of that.
Simon Forman
(As others have already pointed out this would print 1, not 3.)
> >>> print x
>
> [2,1,3]
>
> We often find we need to do manipulations like the above without
> changing the order of the original list, and languages like JS allow
> this. We can't work out how to do this in python though, other than
> duplicating the list, sorting, reversing, then discarding.
In this case you can use:
>>> x = [2,1,3]
>>> print min(x)
1
(There's also a max() function.)
> We're not looking to start any arguments or religious wars and we're
> not asking that python be changed into something its not. We'd simply
> like to understand the decision behind the lists and tuple structures.
> We feel that in not "getting" the difference between the two types we
> may be missing out on using these data structures to their full
> potential.
One way to think about tuples (as distinct from lists) is as a short-
hand version of Pascal's 'record' type, or C's 'struct' (with the
caveats that the fields are not named, only indexed, and the types of
the fields are implied by use, not explicitly declared.)
(FWIW: http://en.wikipedia.org/wiki/Record_(computer_science) )
Broadly speaking, lists are useful for things like stacks and queues,
and sorting, while tuples are useful for aggregating heterogeneous
data into coherent units, and you can hash them (provided their
contents are also hashable.)
HTH,
~Simon
Hendrik van Rooyen
The good reason is the immutability, which lets you use
a tuple as a dict key.
- Hendrik
Steven D'Aprano
> Phillip B Oldham <phillip...@gmail.com> writes:
>
>> We often find we need to do manipulations like the above without
>> changing the order of the original list, and languages like JS allow
>> this. We can't work out how to do this in python though, other than
>> duplicating the list, sorting, reversing, then discarding.
I think Phillip is confused if he thinks that other languages can sort a
list without modifying or duplicating the original. If you don't
duplicate the list, how can you sort it without modifying the original?
The only difference is whether the language makes the duplicate for you,
or expects you to do it yourself.
> If you just want a one-liner, and you don't care about speed you can do
> the following (but I don't think this is considered best practice)
>
>>>> x = [2,1,3]
>>>> print list(sorted(x))
The call to list is redundant, as sorted() already creates a list.
But that's the wrong solution to the problem. The OP wants the largest
(or smallest) item, which he expects to get by sorting, then grabbing the
first element:
sorted(alist)[0]
That requires sorting the entire list, only to throw away everything
except the first item. A better solution is to use min() and max(),
neither of which sort the list, so they are much faster.
--
Steven
Steven D'Aprano
Suppose you could do this:
key = [1, 2]
adict = {key: 'x', [1, 1]: 'y'}
This would work as expected:
adict[ [1, 2] ]
=> returns 'x'
adict[ [1, 1] ]
=> returns 'y'
But what happens if you mutate the key?
key[1] = 0
adict[ [1, 2] ]
=> raises KeyError
Okay, that's bad. What's even worse is this:
key[1] = 1
adict[ [1, 1] ]
=> should it return 'x' or 'y'?
The solution is to disallow mutable objects like lists from being used as
keys in dicts, and to allow immutable list-like tuples instead.
--
Steven
Duncan Booth
> But that's the wrong solution to the problem. The OP wants the largest
> (or smallest) item, which he expects to get by sorting, then grabbing
> the first element:
>
> sorted(alist)[0]
>
> That requires sorting the entire list, only to throw away everything
> except the first item. A better solution is to use min() and max(),
> neither of which sort the list, so they are much faster.
>
And if they want the n smallest or largest items (where n is significantly
less than the length of the list[*]) they might consider using
heapq.nsmallest() and heapq.nlargest()
I find it interesting that the heapq functions tell you in the
documentation that they aren't suitable for use where n==1 or where n is
near the total size of the sequence whereas random.sample() chooses what it
thinks is the best algorithm based on the input. At the very least I would
have thought the heapq functions could check for n==1 and call min/max when
it is.
[*] Some quick playing with timeit seems to indicate that with a list of
200 integers nsmallest is fastest (by a very small margin) when n==2 and
n==3 but is beaten by sorted[:n] when n==4, so I guess you need a
reasonably long list to make it worth not sorting it: with list of 20,000
integers it isn't worth sorting unless you want more than about 700 items.
Inky 788
wrote:
Thanks for the reply Hendrik (and Steven (other reply)). Perhaps I'm
just not sophisticated enough, but I've never wanted to use a list/
tuple as a dict key. This sounds like obscure usage, and a bit
contrived as a reason for having *both* lists and tuples.
Luis Alberto Zarrabeitia Gomez
Quoting Inky 788 <ink...@gmail.com>:
> > The good reason is the immutability, which lets you use
> > a tuple as a dict key. �
>
> Thanks for the reply Hendrik (and Steven (other reply)). Perhaps I'm
> just not sophisticated enough, but I've never wanted to use a list/
> tuple as a dict key. This sounds like obscure usage, and a bit
> contrived as a reason for having *both* lists and tuples.
I don't seem to understand your definition of obscure and contrived. It seems
that you got late to this thread, and you missed the examples. I'd suggest you
to go back on this thread and look at them.
heights = {}
heights[1,2] = 5
heights[1,3] = 7
heights[3,5] = 1
addresses[lastname, firstname] = my_home_address
census[location, age] = census.get((location, age), 0) + 1
All those are using tuples as dict keys.
Regards,
--
Luis Zarrabeitia
Facultad de Matem�tica y Computaci�n, UH
http://profesores.matcom.uh.cu/~kyrie
--
Participe en Universidad 2010, del 8 al 12 de febrero de 2010
La Habana, Cuba
http://www.universidad2010.cu
Gabriel Genellina
Many people posted useful examples of tuples as dictionary keys in this
thread. Just to add another one (emulate SQL GROUP BY):
ope_by_dept = defaultdict(int)
total_times = defaultdict(float)
for dept_name, ope_name, ope_date, engineer in list_of_operations:
ope_by_dept[dept_name, ope_start.month] += 1
total_times[dept_name, engineer] += ope_end - ope_start
print "Operations per department per month"
for dept_name, month in sorted(ope_by_dept):
print dept_name, month, ope_by_dept[dept_name, month]
--
Gabriel Genellina
Hendrik van Rooyen
Steven showed why you cannot have a mutable thing
as a key in a dict.
if you think it is contrived, then please consider how you would
keep track of say the colour of a pixel on a screen at position
(x,y) - this is about the simplest "natural" tuple format and
example.
There are other equally valid examples, as has been pointed
out. (may have been in another thread - am a bit confused
about that right now)
- Hendrik
Tim Rowe
> Thanks for the reply Hendrik (and Steven (other reply)). Perhaps I'm
> just not sophisticated enough, but I've never wanted to use a list/
> tuple as a dict key. This sounds like obscure usage, and a bit
> contrived as a reason for having *both* lists and tuples.
If you are used to working in a language that doesn't allow it then
you'll probably carry on using the work-arounds that you have always
used. It almost certainly only seems obscure because you're not
considering it when it would be a natural solution. In a language that
builds *very* heavily on the concept of dictionaries it's not obscure
at all!
--
Tim Rowe
Mark Lawrence
> My colleagues and I have been working with python for around 6 months
> now, and while we love a lot of what python has done for us and what
> it enables us to do some of the decisions behind such certain
> data-types and their related methods baffle us slightly (when compared
> to the decisions made in other, similarly powerful languages).
>
> Specifically the "differences" between lists and tuples have us
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
>
[rest of original snipped as already discussed]
Sorry if this has been discussed and I've missed it, but how about
memory allocation. An immutable tuple has a fixed memory allocation
whereas that for the mutable list must be liable to change. You might
like to look at the recent thread on this ng 'List insertion cost' and
follow the links to Raymond Hettinger's power point presentation.
Kindest regards.
Mark Lawrence.
Aahz
Mark Lawrence <bream...@yahoo.co.uk> wrote:
>
>Sorry if this has been discussed and I've missed it, but how about
>memory allocation. An immutable tuple has a fixed memory allocation
>whereas that for the mutable list must be liable to change. You might
>like to look at the recent thread on this ng 'List insertion cost' and
>follow the links to Raymond Hettinger's power point presentation.
Not only that, but lists use more memory, period, for any given number of
elements because the overallocation permits amortized constant time for
appends.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22
Beni Cherniavsky
wrote:
wrote:
Dict could allow mutable objects as keys by comparing *by value*,
making a copy on insertion and hashing the current value on lookup.
Prior art: the 2.3 sets module allows mutable Sets as elements in
Sets, by making ImmutableSet copies on insertion, and hashing Sets as
if they are temporarily immutable on lookup.
This inspired PEP 351 and ambitious proposals to expand the approach
to all Python with a copy-on-write scheme. But these ideas were
rejected, and the 2.4 builtin sets only allow frozenset elements.
Half the reason is technical: copy-on-write and harder than it sounds,
and without it you pay a performance price.
But the deeper reason is style: immutable types are convenient!
The allow a pure-functional style of code, which can be simpler.
Of course, sometimes an imperative style is simpler. Depends on the
problem.
My rule of thumb:
- Use mutable lists when you are likely to modify individual elements.
- Use immutable tuples when you are likely to replace the whole thing.
(In practice, this boils down to similar usage to the "official" rule
of thumb that lists are for homogenous data, and tuples for
heterogenous records.)
Inky 788
wrote:
> On Wednesday 22 July 2009 16:36:51 Inky 788 wrote:
>
> > On Jul 22, 2:36 am, Hendrik van Rooyen <hend...@microcorp.co.za>
>
> > wrote:
> > > The good reason is the immutability, which lets you use
> > > a tuple as a dict key.
>
> > Thanks for the reply Hendrik (and Steven (other reply)). Perhaps I'm
> > just not sophisticated enough, but I've never wanted to use a list/
> > tuple as a dict key. This sounds like obscure usage, and a bit
> > contrived as a reason for having *both* lists and tuples.
>
> Steven showed why you cannot have a mutable thing
> as a key in a dict.
>
> if you think it is contrived, then please consider how you would
> keep track of say the colour of a pixel on a screen at position
> (x,y) - this is about the simplest "natural" tuple format and
> example.
My guess is that this is probably the way most people do it:
~~~~
#!/usr/bin/env python
import sys
import random
if len( sys.argv ) != 3:
print "Please pass exactly 2 ints. Exiting."
sys.exit(1)
NUM_COLUMNS = int( sys.argv[1] )
NUM_ROWS = int( sys.argv[2] )
print "Making array of %s columns by %s rows." % (NUM_COLUMNS,
NUM_ROWS)
def rand():
return int( 255 * random.random())
def make_a_pixel():
# red green blue
return [rand(), rand(), rand()]
def make_a_row(num_columns):
temp_row = []
for i in range(num_columns):
temp_row.append( make_a_pixel() )
return temp_row
def make_array_of_pixels(num_columns, num_rows):
rows = []
for i in range(num_rows):
rows.append( make_a_row(num_columns) )
return rows
def show_pixels(pixel_array):
for row in pixel_array:
for pixel in row:
print pixel, ' ',
print
rows_of_pixels = make_array_of_pixels(NUM_COLUMNS, NUM_ROWS)
show_pixels(rows_of_pixels)
~~~~
Uncle Roastie
> My colleagues and I have been working with python for around 6 months
> now, and while we love a lot of what python has done for us and what
> it enables us to do some of the decisions behind such certain
> data-types and their related methods baffle us slightly (when compared
> to the decisions made in other, similarly powerful languages).
>
> Specifically the "differences" between lists and tuples have us
> confused and have caused many "discussions" in the office. We
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
>
>
> >>> x = [2,1,3]
> >>> x.sort()
> >>> print x
>
> [1, 2, 3]
>
> Now, if the sort operations were unable to affect the original
> structure of the list (as in JavaScript) you'd effectively have a
> tuple which you could add/remove from, and the example above would
> look more like:
>
> >>> x = [2,1,3]
> >>> print x.sort()
> [1, 2, 3]
> >>> print x
>
> [2,1,3]
>
> This make a lot more sense to us, and follows the convention from
> other languages. It would also mean chaining methods to manipulate
> lists would be easier:
>
> >>> x = [2,1,3]
> >>> print x.sort()[0]
> 3
>
> [2,1,3]
>
> We often find we need to do manipulations like the above without
> changing the order of the original list, and languages like JS allow
> this. We can't work out how to do this in python though, other than
> duplicating the list, sorting, reversing, then discarding.
>
> not asking that python be changed into something its not. We'd simply
> like to understand the decision behind the lists and tuple structures.
> We feel that in not "getting" the difference between the two types we
> may be missing out on using these data structures to their full
> potential.
A tuple can be used like a struct in C - the number of fields is meant
to
be fixed and should not be dynamically changed.
Hendrik van Rooyen
> On Jul 23, 3:42 am, Hendrik van Rooyen <hend...@microcorp.co.za>
>
> wrote:
Good grief! - Not a dictionary in sight!
I had something in mind where point was an (x,y) tuple,
and point_colours was a dict of possibly named points
with a list of RGB values.
(The colour can change, the point is fixed)
If I were doing it your way, I would use the Array module's Array.
But then, strictly speaking it would not be your way any more,
would it?
- Hendrik
John Nagle
An interesting issue is Python objects, which are always mutable.
A "dict" of Python objects is allowed, but doesn't consider
the contents of the objects, just their identity (address). Only
built-in types are immutable; one cannot create a class of immutable objects.
So things like "frozenset" had to be built into the language.
A tuple is really a frozen list. Arguably, frozen objects
should have been a general concept. Conceptually, they're
simple - once "__init__" has run, there can be no more changes
to fields of the object.
Compare the C++ approach, where you can have a "map" of
any object that defines the "<" operator. Python has
"__cmp__" for objects, but built-in dictionaries don't use it.
Of course, one could create a "map" class in Python that
uses "__cmp__", which would allow dictionary-type operations
on arbitrary objects.
There are some interesting uses for immutable objects in
multithreaded programs. They can be shared between threads
without locking. If it weren't for the infamous Global
Interpreter Lock, which basically limits Python programs to using
one CPU at a time, Python would have to deal with locking
in a more advanced way. A system where immutable objects
can be shared and passed between threads, and mutable objects
must be "synchronized" (in the Java sense) to be shared would
be useful.
John Nagle
Steven D'Aprano
> An interesting issue is Python objects, which are always mutable.
> A "dict" of Python objects is allowed, but doesn't consider the contents
> of the objects, just their identity (address). Only built-in types are
> immutable; one cannot create a class of immutable objects.
Yes you can, for some definition of "can":
http://northernplanets.blogspot.com/2007/01/immutable-instances-in-python.html
Admittedly pure Python objects are only "cooperatively immutable". The
immutability relies on the caller not going out of its way to break the
instance, so you can mutate it if you work at it. One could, I suppose,
try putting in complicated tricks to prevent that, but anything written
in Python can ultimately be modified in Python.
--
Steven
John Nagle
> On Sun, 26 Jul 2009 11:24:48 -0700, John Nagle wrote:
>
>> An interesting issue is Python objects, which are always mutable.
>> A "dict" of Python objects is allowed, but doesn't consider the contents
>> of the objects, just their identity (address). Only built-in types are
>> immutable; one cannot create a class of immutable objects.
>
> Yes you can, for some definition of "can":
>
> http://northernplanets.blogspot.com/2007/01/immutable-instances-in-python.html
>
>
> Admittedly pure Python objects are only "cooperatively immutable".
Right. I've been thinking about this as a way of improving
concurrency handling. The general idea is that objects shared
across threads would have to be either be immutable or synchronized.
Regular objects would be limited to a single thread. It's a path
to multithread programs without a global lock. Needs more work
to become a serious proposal. (It may not be worth the trouble;
a few years ago we were hearing about how 64-core CPUs were going
to become mainstream, but instead, the industry is going in the
direction of the same compute power for less money and with
less power consumption.)
John Nagle
Benjamin Kaplan
Python dictionaries are hash maps so they use hashes, not comparisons,
to store objects. By default the hash is equal to the object's
identity but all you need to do to change it is define your own
__hash__ method. If you were to make a C++ hash map, it wouldn't use
comparisons either.
>
> There are some interesting uses for immutable objects in
> multithreaded programs. They can be shared between threads
> without locking. If it weren't for the infamous Global
> Interpreter Lock, which basically limits Python programs to using
> one CPU at a time, Python would have to deal with locking
> in a more advanced way. A system where immutable objects
> can be shared and passed between threads, and mutable objects
> must be "synchronized" (in the Java sense) to be shared would
> be useful.
>
> John Nagle
Dave Angel
>>
>>> On Jul 22, 9:36 am, Hendrik van Rooyen <hend...@microcorp.co.za>
>>> wrote:
>>>
>>>> On Tuesday 21 July 2009 15:49:59 Inky 788 wrote:
>>> problem.
>>>
>> An interesting issue is Python objects, which are always mutable.
>> A "dict" of Python objects is allowed, but doesn't consider
>> the contents of the objects, just their identity (address). Only
>> built-in types are immutable; one cannot create a class of immutable
>> objects.
>> So things like "frozenset" had to be built into the language.
>> A tuple is really a frozen list. Arguably, frozen objects
>> should have been a general concept. Conceptually, they're
>> simple - once "__init__" has run, there can be no more changes
>> to fields of the object.
>>
>> Compare the C++ approach, where you can have a "map" of
>> any object that defines the "<" operator. Python has
>> "__cmp__" for objects, but built-in dictionaries don't use it.
>> Of course, one could create a "map" class in Python that
>> uses "__cmp__", which would allow dictionary-type operations
>> on arbitrary objects.
>>
>
>
> to store objects. By default the hash is equal to the object's
> identity but all you need to do to change it is define your own
> __hash__ method. If you were to make a C++ hash map, it wouldn't use
> comparisons either.
>
>
>
I think you'll find that to make a proper dict of user-objects, you need
to define not only __hash__(), but also __eq__() methods. A hash is
only a starting place. In case of collision, the dict class calls
another method to make sure.
And of course, for the dict to function reasonably, you have to make
sure that the hash and eq functions return consistent results, at least
pretending that the objects involved are immutable.
DaveA
Joshua Bronson
> I find it interesting that the heapq functions tell you in the
> documentation that they aren't suitable for use where n==1 or where n is
> near the total size of the sequence whereas random.sample() chooses what it
> thinks is the best algorithm based on the input. At the very least I would
> have thought the heapq functions could check for n==1 and call min/max when
> it is.
I've had the same thought myself a number of times, so I filed a bug:
http://bugs.python.org/issue6614
Raymond Hettinger
> Steven D'Aprano <ste...@REMOVE.THIS.cybersource.com.au> wrote:
> > But that's the wrong solution to the problem. The OP wants the largest
> > (or smallest) item, which he expects to get by sorting, then grabbing
> > the first element:
>
> > sorted(alist)[0]
>
> > That requires sorting the entire list, only to throw away everything
> > except the first item. A better solution is to use min() and max(),
> > neither of which sort the list, so they are much faster.
>
> And if they want the n smallest or largest items (where n is significantly
> less than the length of the list[*]) they might consider using
> heapq.nsmallest() and heapq.nlargest()
>
> I find it interesting that the heapq functions tell you in the
> documentation that they aren't suitable for use where n==1 or where n is
> near the total size of the sequence whereas random.sample() chooses what it
> thinks is the best algorithm based on the input. At the very least I would
> have thought the heapq functions could check for n==1 and call min/max when
> it is.
The heapq.py code in Py2.7 and Py3.1 already does some automatic
algorithm selection:
http://svn.python.org/view/python/branches/release31-maint/Lib/heapq.py?revision=73579&view=markup
The automatic seletion of alternatives only occurs in clear-cut cases
(such as n==1
or where n==len(iterable) when the iterable has a known length). For
the
remaining cases, the switchover point from heapq to sorted needs a
programmer's judgment based on whether the input iterable has a known
length, the cost of comparison function, and whether input is already
partially ordered.
The advice in the docs helps the reader understand the
relationships between min, max, nsmallest, nlargest, and sorted.
Raymond
Raymond Hettinger
> Specifically the "differences" between lists and tuples have us
> confused and have caused many "discussions" in the office. We
> understand that lists are mutable and tuples are not, but we're a
> little lost as to why the two were kept separate from the start. They
> both perform a very similar job as far as we can tell.
The underlying C code for the two is substantially the same. In terms
of code, tuples are in effect just immutable lists that have the
added
feature of being hashable (and therefore usable as dictionary keys or
elements of sets).
Beyond the mutable/hashable distinction, there is an important
philosophical distinction articulated by Guido. He deems tuples
to be useful for struct like groupings of non-homogenous fields
and lists to be useful for sequences of homogenous data suitable
for looping.
While nothing in the list/tuple code requires you to make that
distinction,
it is important because that philosophy pervades the language. If you
follow Guido's direction, you'll find that the various parts of the
language fit together better. Do otherwise and you'll be going
against
the grain.
Raymond
P.S. The C code for lists and tuples have a slightly difference
internal
structure that makes tuples a little more space efficient (since their
size is known at the outset and doesn't change during use).
Emmanuel Surleau
> On Jul 20, 9:27 am, Phillip B Oldham <phillip.old...@gmail.com> wrote:
> > Specifically the "differences" between lists and tuples have us
> > confused and have caused many "discussions" in the office. We
> > understand that lists are mutable and tuples are not, but we're a
> > little lost as to why the two were kept separate from the start. They
> > both perform a very similar job as far as we can tell.
>
> The underlying C code for the two is substantially the same. In terms
> of code, tuples are in effect just immutable lists that have the
> added
> feature of being hashable (and therefore usable as dictionary keys or
> elements of sets).
>
> Beyond the mutable/hashable distinction, there is an important
> philosophical distinction articulated by Guido. He deems tuples
> to be useful for struct like groupings of non-homogenous fields
> and lists to be useful for sequences of homogenous data suitable
> for looping.
>
> While nothing in the list/tuple code requires you to make that
> distinction,
> it is important because that philosophy pervades the language. If you
> follow Guido's direction, you'll find that the various parts of the
> language fit together better. Do otherwise and you'll be going
> against
> the grain.
I might be wrong, but I get the impression that many people don't indeed "get
it" and use tuples as static arrays and lists when the array needs to be
dynamically resized. This is certainly how I use them as well.
This would tend to show that Guido's notion here was not particularly
intuitive.
Cheers,
Emm
Masklinn
> On Friday 31 July 2009 19:49:04 Raymond Hettinger wrote:
> indeed "get
> it" and use tuples as static arrays and lists when the array needs
> to be
> dynamically resized. This is certainly how I use them as well.
>
> This would tend to show that Guido's notion here was not particularly
> intuitive.
tuples (which are mostly functional, and in which tuples are *never*
sequences/iterables). At the end of the day, and if Guido's intention
truly was what Raymond says, implementing tuples as immutable sequence
was a mistake. And the documentation is to blame as well: in it,
tuples are clearly described as a *sequence* type, not a *structure*
type.
Terry Reedy
>
>> Beyond the mutable/hashable distinction, there is an important
>> philosophical distinction articulated by Guido. He deems tuples to
>> be useful for struct like groupings of non-homogenous fields and
>> lists to be useful for sequences of homogenous data suitable for
>> looping.
I think the use of the with 'homegeneous' in this context is wrong
because 1) it is ambiguous or even sometimes not the case, and 2) it
misleads. Everything is an object, so all collections are in that sense
homogeneous. If the meaning is restricted to type(ob), than a list
'mixing' ints and floats would be non-homogeneous and not 'legitimate',
but Guido does not really mean that.
The word tuple comes from relational databases as a generalization of
single, double, triple, quadruple, quintuple, sextuple, sestuple,
octuple, etc. A tuple is a data record with a fixed number of fields
with individual meaning. There is nothing about homogeneity of data type
in that definition. A triple of floats is legitimately a tuple when each
is a coordinate (the individual meanings). In other contexts, the same
triple might properly be a list (such as of heights of people
arbitrarily ordered).
>> While nothing in the list/tuple code requires you to make that
>> distinction,
which needs to be properly articulated...
>> it is important because that philosophy pervades the language. If
>> you follow Guido's direction, you'll find that the various parts of
>> the language fit together better. Do otherwise and you'll be going
>> against the grain.
Understanding the idea certainly helps understanding the design.
For instance, it explains why no tuple comprehensions.
> I might be wrong, but I get the impression that many people don't
> indeed "get it" and use tuples as static arrays and lists when the
> array needs to be dynamically resized. This is certainly how I use
> them as well.
You also *need* lists if you want to dynamically re-arrange or replace.
If you *must not* re-arrange or edit, then a tuple will prevent that.
I would also refer back to the idea of tuple as database record.
But a list is needed to edit without copying, so lists can be used for
writable database records.
I believe tuple(iterable) for indefinite-length iterables is somewhat
equivalent to tuple(list(iterable)), so there is usually no point to
constructing a tuple that way from such an input.
For collections defined by constants, there is a real advantage to using
tuples, since tuple constants are compiled when the bytecode is
compiled, rather than being dynamically constructed. Inside an inner
loop, this might make enough time difference that "Practicality beats
purity" would apply.
> This would tend to show that Guido's notion here was not particularly
> intuitive.
The problem is that it *is* intuitive, on his part, and usually not well
explained rationally.
Terry Jan Reedy
Emmanuel Surleau
> Emmanuel Surleau wrote:
> >> Beyond the mutable/hashable distinction, there is an important
> >> philosophical distinction articulated by Guido. He deems tuples to
> >> be useful for struct like groupings of non-homogenous fields and
> >> lists to be useful for sequences of homogenous data suitable for
> >> looping.
>
> I think the use of the with 'homegeneous' in this context is wrong
> because 1) it is ambiguous or even sometimes not the case, and 2) it
> misleads. Everything is an object, so all collections are in that sense
> homogeneous. If the meaning is restricted to type(ob), than a list
> 'mixing' ints and floats would be non-homogeneous and not 'legitimate',
> but Guido does not really mean that.
>
> The word tuple comes from relational databases as a generalization of
> single, double, triple, quadruple, quintuple, sextuple, sestuple,
> octuple, etc. A tuple is a data record with a fixed number of fields
> with individual meaning. There is nothing about homogeneity of data type
> in that definition. A triple of floats is legitimately a tuple when each
> is a coordinate (the individual meanings). In other contexts, the same
> triple might properly be a list (such as of heights of people
> arbitrarily ordered).
My understanding is that, in this context, it's not so much data types which
are heterogeneous, but the semantic meaning of the data. For instance, a tuple
containing (first_name, last_name, address) would be a "legitimate" tuple, but
not a tuple containing (address, address, address), which, if we follow
Guido's philosophy, ought to be represented as a list.
Whether including the distinction in the language offers any practical benefit
is questionable.
[snip]
> > This would tend to show that Guido's notion here was not particularly
> > intuitive.
>
> The problem is that it *is* intuitive, on his part, and usually not well
> explained rationally.
Excellent point.
Cheers,
Emm
Raymond Hettinger
> > distinction,
> > it is important because that philosophy pervades the language. If you
> > follow Guido's direction, you'll find that the various parts of the
> > language fit together better. Do otherwise and you'll be going
> > against
> > the grain.
>
> I might be wrong, but I get the impression that many people don't indeed "get
> it" and use tuples as static arrays and lists when the array needs to be
> dynamically resized. This is certainly how I use them as well.
>
> This would tend to show that Guido's notion here was not particularly
> intuitive.
No, it just shows that they are not Dutch ;-)
IIRC, Aahz once made an eloquent statement about how languages aren't
intuitive because different people have different intuitions. The
fact
is that every language has its own set of ways. People usually pick-
up
on what is Pythonic in a short order. The problem comes when people
decide-in-advance how the language ought to be -- it takes them longer
to pick-up the culture and idioms. More than one person here has
observed that the time to learn to program Pythonically is inversely
proportional to their experience in Java.
It is perfectly okay for you to use tuples for statically sized
arrays.
That isn't wrong. It is against then grain though. Guido articulated
his concept early and many of his minions (me included) have designed
the APIs and optimizations to flow nicely with his preferred way of
doing things.
Raymond
Raymond Hettinger
> A tuple is really a frozen list. Arguably, frozen objects
> should have been a general concept. Conceptually, they're
> simple - once "__init__" has run, there can be no more changes
> to fields of the object.
I would argue that freezing and thawing (making the container
statefull)
is the proverbial road to hell paved with good intentions. It is
roughly in the same category as dynamic scoping -- programs become
harder to analyze for correctness -- code that used to work can
break because something froze it along the way (perhaps to use it
as a key for dict or set in a caching function) or because something
unfroze it (so it could mutate a value).
Raymond
Gabriel Genellina
<emmanuel...@gmail.com> escribi�:
> On Friday 31 July 2009 21:55:11 Terry Reedy wrote:
>> The word tuple comes from relational databases as a generalization of
>> single, double, triple, quadruple, quintuple, sextuple, sestuple,
>> octuple, etc. A tuple is a data record with a fixed number of fields
>> with individual meaning. There is nothing about homogeneity of data type
>> in that definition. A triple of floats is legitimately a tuple when each
>> is a coordinate (the individual meanings). In other contexts, the same
>> triple might properly be a list (such as of heights of people
>> arbitrarily ordered).
>
> My understanding is that, in this context, it's not so much data types
> which
> are heterogeneous, but the semantic meaning of the data. For instance, a
> tuple
> containing (first_name, last_name, address) would be a "legitimate"
> tuple, but
> not a tuple containing (address, address, address), which, if we follow
> Guido's philosophy, ought to be represented as a list.
Note that years ago the distinction was much stronger: tuples had NO
methods at all. All you could do with a tuple was: len(t), t[i], t1+t2,
t*n, e in t, for e in t
Being so "crippled", thinking of tuples just as immutable lists probably
wasn't so natural.
--
Gabriel Genellina
sturlamolden
> We're not looking to start any arguments or religious wars and we're
> not asking that python be changed into something its not. We'd simply
> like to understand the decision behind the lists and tuple structures.
> We feel that in not "getting" the difference between the two types we
> may be missing out on using these data structures to their full
> potential.
A crude simplification would be:
- A Python programmer will use a tuple where a C programmer will use a
struct.
- Python and C programmers use lists and arrays similarly.
Tuples are used for passing arguments to and from a function. Common
use of tuples include multiple return values and optional arguments
(*args).
It has already been mentioned that tuples are immutable and can be
used as dictionary keys.
sturlamolden
> More than one person here has
> observed that the time to learn to program Pythonically is inversely
> proportional to their experience in Java.
I believe it is opposite. The longer the Java experience, the longer
it takes to program pythonically. The correlation is not inverse. In
particular, programmers accustomed to Java or C++ seem to use for-
loops awfully lot. And when they do, they always loop over a sequence
of integers, and use these as array indices. They consequently use a
class where a tuple would be more appropriate, never return multiple
values from functions, etc. Due to their coding style, they produce
programs that are notoriously inefficient, and blame this on 'Python
being slow'. Unlearning the idioms from Java takes a while. Python is
not Java and should not be programmed as such.
On the other hand, I found that my experience with MATLAB helped me a
lot on picking up a pythonical style quickly.
Nobody
>>�More than one person here has
>> observed that the time to learn to program Pythonically is inversely
>> proportional to their experience in Java.
>
> I believe it is opposite. The longer the Java experience, the longer
> it takes to program pythonically. The correlation is not inverse. In
> particular, programmers accustomed to Java or C++ seem to use for-
> loops awfully lot. And when they do, they always loop over a sequence
> of integers, and use these as array indices.
It would be more accurate to say that of programmers who are *only*
accustomed to Java or C++. I've been using C for 80-90% of my code for 25
years, and I wouldn't normally loop over the indices. But then the other
10-20% is Lisp, Tcl, Haskell, PostScript and, more recently, Python.
OTOH, using a "for" loop when you could use a generator means less work
when you need to make a minor change and a generator is no longer
sufficient.
sturlamolden
> OTOH, using a "for" loop when you could use a generator means less work
> when you need to make a minor change and a generator is no longer
> sufficient.
It's not just that. It is e.g. using a for loop and indexes instead of
a slice. E.g.
for i in range(10): # Java idiom
a[i] = b[i+1]
instead of
a[:10] = b[1:11] # Pythonic
sturlamolden
> It's intuitive if you come to Python knowing other languages with
> tuples (which are mostly functional, and in which tuples are *never*
> sequences/iterables). At the end of the day, and if Guido's intention
> truly was what Raymond says, implementing tuples as immutable sequence
> was a mistake. And the documentation is to blame as well: in it,
> tuples are clearly described as a *sequence* type, not a *structure*
> type.
That depends on the readers' preconceptions: Only when the reader have
knowledge of C structs, Pascal records, or Java classes should this
matter. The naïve reader will not have preconceptions about a
difference between "structure types" and "sequence types". A more
advanced C programmer will also know that a struct is a form of a
sequence in C: you can e.g. memcpy any struct to a char array.
Dave Angel
I think collections.namedtuple, introduced in Python 2.6, might help
bridge the gap between a traditional tuple and a C struct.
Marcus Wanner
> - Python and C programmers use lists and arrays similarly.
I'm guessing that's because of the brackets...
Marcus
John Nagle
> sturlamolden wrote:
>> On 20 Jul, 18:27, Phillip B Oldham <phillip.old...@gmail.com> wrote:
>> Tuples are used for passing arguments to and from a function. Common
>> use of tuples include multiple return values and optional arguments
>> (*args).
That's from Mesa, the Xerox PARC language of the 1970s.
Mesa used tuples for subroutine arguments in a very straightforward
way. Every function took one tuple as an argument, written as parameters
in parentheses separated by commas. Every function returned a tuple
as an argument. This had a nice symmetry; function outputs and
function inputs had the same form. Mesa was the first language
to break through the "single return value" syntax problem.
Python doesn't go that far.
John Nagle
alex23
> Every function returned a tuple as an argument. This had a nice
> symmetry; function outputs and function inputs had the same form.
> Mesa was the first language to break through the "single return
> value" syntax problem.
>
> Python doesn't go that far.
I assume here you're referring to the former point and not the latter?
greg
> Mesa used tuples for subroutine arguments in a very straightforward
> way. Every function took one tuple as an argument
>
I believe that a very early version of Python did do
something like that, but it was found to be a bad idea,
because there was an ambiguity between multiple args and
a single arg that happened to be a tuple.
This was before keyword arguments existed -- they would
have been hard to incorporate into the old scheme.
--
Greg
Masklinn
> Dave Angel wrote:
>> sturlamolden wrote:
>>> On 20 Jul, 18:27, Phillip B Oldham <phillip.old...@gmail.com> wrote:
>>> Tuples are used for passing arguments to and from a function. Common
>>> use of tuples include multiple return values and optional arguments
>>> (*args).
>
> That's from Mesa, the Xerox PARC language of the 1970s.
>
> Mesa used tuples for subroutine arguments in a very straightforward
> way. Every function took one tuple as an argument, written as
> parameters
> in parentheses separated by commas.
same: uncurried functions really take a tuple as single argument
rather than multiple arguments, using pattern matching to make it look
like multiple arguments. Then again, most of them seem to default to
curried functions these days, which is nice.