list() coercion
Ian Bicking
specifically __len__. These methods are rather expensive (they result
in database calls, COUNT(*) to be specific), but cheaper than iterating
over the object. Sometimes it is useful to create a list from the
iterator, using list(). However, list() seems to call the object's
__len__, I imagine to pre-allocate space. This is a problem, as
pre-allocation saves much less than is spent doing __len__.
Is there a way I can keep this from happening? Maybe something list()
tries first that I can make fail. (I notice list() catches any
exceptions in __len__ and then will just skip that step)
Ian
Delaney, Timothy C (Timothy)
>
> Is there a way I can keep this from happening? Maybe something list()
> tries first that I can make fail. (I notice list() catches any
> exceptions in __len__ and then will just skip that step)
Simplest thing is probably:
ll = MyListLikeObject()
li = iter(ll)
l = list(li)
i.e. explicitly create an iterator (which doesn't have a __len__) and create the list from that.
OTOH, if the problem is that creating the iterator is causing the problem (calling __len__), you may need to create a proxy object that doesn't have a __len__ and call list() on that.
Tim Delaney
Greg Ewing (using news.cis.dfn.de)
> However, list() seems to call the object's
> __len__,
>
Give your object an __iter__ method that returns
itself.
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Ian Bicking
> Ian Bicking wrote:
> > However, list() seems to call the object's
> > __len__,
> >
> > Is there a way I can keep this from happening?
>
> Give your object an __iter__ method that returns
> itself.
I'm not clear on how that will help...?
It already does have an __iter__ method, but it returns a separate
iterator.
Ian
Bob Gailer
I have just read the docs 2.2.5 Iterator Types. Unfortunately this page
seems to be written for someone who already understands the page. Is there
any other explanation or examples?
Bob Gailer
bga...@alum.rpi.edu
303 442 2625
Alan Kennedy
> I have just read the docs 2.2.5 Iterator Types. Unfortunately this page
> seems to be written for someone who already understands the page. Is
> there any other explanation or examples?
Try A. M. Kuchling's "What's new in Python".
http://www.python.org/doc/2.2.1/whatsnew/node4.html
And if you want to look under the covers
http://www.python.org/peps/pep-0234.html
HTH,
--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
Alan Kennedy
>>> However, list() seems to call the object's
>>> __len__,
>>>
>>> Is there a way I can keep this from happening?
Greg Ewing:
>> Give your object an __iter__ method that returns
>> itself.
Ian Bicking wrote:
> I'm not clear on how that will help...?
>
> It already does have an __iter__ method, but it returns a separate
> iterator.
Just to be explicitly clear
1. You're retrieving record sets from a database.
2. You want to build a list from the database results, one list entry
for each result row.
3. You don't want to have the list preallocated (thus requiring
invocation of __len__()), because the time saving is not worth it
(compared to the cost of the SQL count() operation), since you're
going to be iterating over the record set anyway.
Therefore, use an iterator to build up the list. This will not call
__len__(). Instead, the list will continually be appended to, until
the iterator raise StopIteration. The list may be re-allocated
multiple times, as the number of records retrieved exceeds the
allocation unit size of lists. But this will likely still be lower
cost than your SQL count().
The object returned from the __iter__() method should have a .next()
method which returns the next row in your result set. So if you have
implemented a .next() on your result set object, define your
__.iter__() method as follows
class ResultSet:
def next(self):
"Pseudocode"
result = databasecursor.fetchone()
if result:
return result
else:
raise StopIteration
def __iter__(self):
return self
Have I understood the problem correctly?
Raymond Hettinger
"Ian Bicking" <ia...@colorstudy.com> wrote in message
news:mailman.1058400718...@python.org...
Instead of:
list(yourobj)
use:
list(iter(yourobj))
If that doesn't help, create your own wrapper:
def myiter(it):
for elem in it:
yield it
list(myiter(yourobj))
This idea is to provide 'list' with a wrapper that only supplies
the iter methods and not the len method.
Raymond Hettinger
Bob Gailer
>Bob Gailer wrote:
>
> > I have just read the docs 2.2.5 Iterator Types. Unfortunately this page
> > seems to be written for someone who already understands the page. Is
> > there any other explanation or examples?
>
>Try A. M. Kuchling's "What's new in Python".
>
>http://www.python.org/doc/2.2.1/whatsnew/node4.html
That still sounds like it was written for someone who already understands
the material. What I'd like is something that I could read once, that would
define each new term, and would leave me a clear understanding of the topic.
Perhaps I will attempt to digest the topic then write the kind of
explanation that I wanted to start with and submit that to the document folks.
I find the same frustration with other Python documentation.
Raymond Hettinger
> I have just read the docs 2.2.5 Iterator Types. Unfortunately this page
> seems to be written for someone who already understands the page. Is there
> any other explanation or examples?
The Py2.3 release candidate goes out tonight.
In it, I've included sections on iterators and generators
in the tutorial. I'm interested to know whether you
find it helpful:
http://www.python.org/dev/doc/devel/tut/node11.html#SECTION001190000000000000000
0
Raymond Hettinger
Ian Bicking
> Instead of:
>
> list(yourobj)
>
> use:
>
> list(iter(yourobj))
Sigh... that's too bad. I'll probably just get rid of the __len__
method instead, as list(yourobj) (list(myobj)?) is a more appealing
idiom than len(somebodysobj).
Ian
Greg Ewing (using news.cis.dfn.de)
> I'm not clear on how that will help...?
>
> It already does have an __iter__ method, but it returns a separate
> iterator.
It turns out that it won't help. I had thought that
calling __len__ would only happen if there were no
__iter__ method, but that seems not to be the case.
The only thing I can think of at the moment is not
to do list(x) at all, but list(iter(x)) instead.