dict.items() vs dict.iteritems and similar questions

[Drew]

When is it appropriate to use dict.items() vs dict.iteritems. Both
seem to work for something like:

for key,val in mydict.items():
print key,val

for key,val in mydict.iteritems():
print key,val

Also, when is it appropriate to use range() vs xrange(). From my
understanding, xrange() essentially gives you an iterator across a
range, so it should be used when iterating. Should you only use
range() when want to physically store the range as a list?

Thanks,
Drew

[Laurent Pointal]

Drew a écrit :

iteritems and xrange only provide values when requested.
items and range build complete list when called.

Both work, you may prefer xrange/iteritems for iteration on large
collections, you may prefer range/items when processing of the result
value explicitly need a list (ex. calculate its length) or when you are
going to manipulate the original container in the loop.

A+

Laurent.

[Drew]

On Mar 14, 11:44 am, Laurent Pointal <laurent.poin...@limsi.fr> wrote:
> Both work, you may prefer xrange/iteritems for iteration on large
> collections, you may prefer range/items when processing of the result
> value explicitly need a list (ex. calculate its length) or when you are
> going to manipulate the original container in the loop.
>
> A+
>
> Laurent.

Laurent -

Extremely helpful, exactly what I was looking for.

Thanks,
Drew

[Shane Geiger]

# Just by looking at the output, it seems pretty obvious that xrange
would be more memory effcient for large ranges:

print "With range():",range(100,200)
print
print "With xrange():",xrange(100,200)

d = {1:2,2:3,3:4}
d.items()
d.iteritems()

# I have been curious to use Pysizer (which requires patching Python)
to demonstrate the difference.

--
Shane Geiger
IT Director
National Council on Economic Education
sge...@ncee.net | 402-438-8958 | http://www.ncee.net

Leading the Campaign for Economic and Financial Literacy

[Leif K-Brooks]

Laurent Pointal wrote:
> Both work, you may prefer xrange/iteritems for iteration on large
> collections, you may prefer range/items when processing of the result
> value explicitly need a list (ex. calculate its length) or when you are
> going to manipulate the original container in the loop.

xrange actually supports len():

>>> len(xrange(10))
10

[sk...@pobox.com]

>> When is it appropriate to use dict.items() vs dict.iteritems.

Laurent> Both work, you may prefer xrange/iteritems for iteration on
Laurent> large collections...

I find "iter<anything>" to be extremely ugly and hope to avoid using them
altogether until they are gone in Py3k.

Skip

[Drew]

Skip -

Ugly, maybe, but don't you take a decent performance hit when loading
the entire dict into memory at once? Especially if the dict is large?

[sk...@pobox.com]

>> I find "iter<anything>" to be extremely ugly and hope to avoid using
>> them altogether until they are gone in Py3k.

Drew> Ugly, maybe, but don't you take a decent performance hit when
Drew> loading the entire dict into memory at once? Especially if the
Drew> dict is large?

Sure, but I try hard to keep my dicts small. ;-)

Skip

[Paul Rubin]

Laurent Pointal <laurent...@limsi.fr> writes:
> Both work, you may prefer xrange/iteritems for iteration on large
> collections, you may prefer range/items when processing of the result
> value explicitly need a list (ex. calculate its length) or when you are
> going to manipulate the original container in the loop.

You can use len(d) if d is a dict.

[Laurent Pointal]

Paul Rubin a écrit :

Yes, as long as you have a semantic relation between the original dict
and the extracted keys. But once you have extracted the keys, and pass
them around your functions, if you miss the relationship, you have
either a list container or a generator. Not considering the case where
original dict is modified between keys extraction and keys usage...

But as we dont know more about OP needs...

A+

Laurent.

[bearoph...@lycos.com]

Laurent Pointal:

> you may prefer range/items when processing of the result
> value explicitly need a list (ex. calculate its length)

Creating a very long list just to know the len of an iterator is
barbaric, so sometimes I use this:

def leniter(iterator):
if hasattr(iterator, "__len__"):
return len(iterator)
nelements = 0
for _ in iterator:
nelements += 1
return nelements

Bye,
bearophile

[Steve Holden]

Of course this is a little like the Heisenberg uncertainty principle if
the iterator has no __len__ attribute - once you know how long it is you
no longer have access to the elements. Or did I miss something?

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007

[Bruno Desthuilliers]

Steve Holden a écrit :

> bearoph...@lycos.com wrote:
>> Laurent Pointal:
>>> you may prefer range/items when processing of the result
>>> value explicitly need a list (ex. calculate its length)
>>
>> Creating a very long list just to know the len of an iterator is
>> barbaric, so sometimes I use this:
>>
>> def leniter(iterator):
>> if hasattr(iterator, "__len__"):
>> return len(iterator)
>> nelements = 0
>> for _ in iterator:
>> nelements += 1
>> return nelements
>>
> Of course this is a little like the Heisenberg uncertainty principle if
> the iterator has no __len__ attribute - once you know how long it is you
> no longer have access to the elements. Or did I miss something?

yes, that's one of the side effects. Another interesting case:

import itertools
it = itertools.cycle(range(10))
print "it has %d elements" % leniter(it)

[bearoph...@lycos.com]

Steve Holden:

> once you know how long it is you
> no longer have access to the elements. Or did I miss something?

Now and then I need to know how many elements there are, and not what
they are, so in those situations storing them isn't necessary.

Bye,
bearophile

[Alex Martelli]

Steve Holden <st...@holdenweb.com> wrote:

> bearoph...@lycos.com wrote:
> > Laurent Pointal:
> >> you may prefer range/items when processing of the result
> >> value explicitly need a list (ex. calculate its length)
> >
> > Creating a very long list just to know the len of an iterator is
> > barbaric, so sometimes I use this:
> >
> > def leniter(iterator):
> > if hasattr(iterator, "__len__"):
> > return len(iterator)
> > nelements = 0
> > for _ in iterator:
> > nelements += 1
> > return nelements
> >
> Of course this is a little like the Heisenberg uncertainty principle if
> the iterator has no __len__ attribute - once you know how long it is you
> no longer have access to the elements. Or did I miss something?

Right. However, "return sum(1 for _ in iterator)" may be a handier way
to express the same desctructive semantics as the last 4 lines here.

Alex

[Duncan Booth]

al...@mac.com (Alex Martelli) wrote:

>> Of course this is a little like the Heisenberg uncertainty principle if
>> the iterator has no __len__ attribute - once you know how long it is you
>> no longer have access to the elements. Or did I miss something?
>
> Right. However, "return sum(1 for _ in iterator)" may be a handier way
> to express the same desctructive semantics as the last 4 lines here.

I think I'd prefer the barbaric:

return len(list(iterator))

since at least it is guaranteed to terminate.

[sk...@pobox.com]

Duncan> I think I'd prefer the barbaric:

Duncan> return len(list(iterator))

Duncan> since at least it is guaranteed to terminate.

Are you sure? There's no guarantee that an iterator will terminate:

len(list(itertools.cycle(range(10))))

Skip

[Marc 'BlackJack' Rintsch]

In <mailman.5140.1173979...@python.org>, skip wrote:

> Are you sure? There's no guarantee that an iterator will terminate:
>
> len(list(itertools.cycle(range(10))))

You have infinite memory? ;-)

Ciao,
Marc 'BlackJack' Rintsch

[bearoph...@lycos.com]

Alex Martelli:

> Right. However, "return sum(1 for _ in iterator)" may be a handier way
> to express the same desctructive semantics as the last 4 lines here.

With the speed tests I have done my version did come out as the faster
one.

Bye,
bearophile

[Duncan Booth]

Marc 'BlackJack' Rintsch <bj_...@gmx.net> wrote:

> In <mailman.5140.1173979...@python.org>, skip wrote:
>
>> Are you sure? There's no guarantee that an iterator will terminate:
>>
>> len(list(itertools.cycle(range(10))))
>
> You have infinite memory? ;-)

Strangely, Skip's example is exactly the one I tested before posting my
claim that it would terminate.