for loop: range() result has too many items
Peng Yu
The following code does not run because range() does not accept a big
number. Is there a way to make the code work. I'm wondering if there
is a way to write a for-loop in python similar to that of C style.
for(int i = 0; i < a_big_number; ++ i)
Regards,
Peng
$ cat for_loop.py
import sys
def foo():
for i in range(sys.maxint):
if i % 100 == 0:
print i
foo()
$ python for_loop.py
Traceback (most recent call last):
File "for_loop.py", line 8, in <module>
foo()
File "for_loop.py", line 4, in foo
for i in range(sys.maxint):
OverflowError: range() result has too many items
Mel
Are you using Python 2.x ??? If you are, range will accept your number, but
it will try to build a list holding sys.maxint integers, and you don't have
enough memory.
Try xrange instead.
Mel.
Steven D'Aprano
> Hi,
>
> The following code does not run because range() does not accept a big
> number.
Incorrect.
>>> range(sys.maxint+2, sys.maxint+5)
[2147483649L, 2147483650L, 2147483651L]
> Is there a way to make the code work. I'm wondering if there is
> a way to write a for-loop in python similar to that of C style.
>
> for(int i = 0; i < a_big_number; ++ i)
An exact translation of that would be a while loop:
i = 0
while i < a_big_number:
i += 1
but that's not actually your problem. Your actual problem is that you're
trying to generate a list containing 2147483647 (sys.maxint) items. Each
item requires 8 bytes, so this will need a single contiguous block of at
least 16 gigabytes.
On my system, when I try it, I get MemoryError. You get OverflowError.
Try instead using a lazy list:
>>> xrange(sys.maxint)
xrange(2147483647)
> def foo():
> for i in range(sys.maxint):
> if i % 100 == 0:
> print i
Rather than iterating through every single integer between 0 and
2147483647, you can skip the ones you don't care about:
for i in xrange(0, sys.maxint, 100):
print i
This will run approximately 100 times faster than your code.
--
Steven
Andre Engels
> Hi,
>
> The following code does not run because range() does not accept a big
> is a way to write a for-loop in python similar to that of C style.
>
> for(int i = 0; i < a_big_number; ++ i)
>
> Peng
>
> $ cat for_loop.py
> import sys
>
> for i in range(sys.maxint):
> if i % 100 == 0:
> print i
>
> $ python for_loop.py
> Traceback (most recent call last):
> File "for_loop.py", line 8, in <module>
> foo()
> File "for_loop.py", line 4, in foo
> for i in range(sys.maxint):
> OverflowError: range() result has too many items
> http://mail.python.org/mailman/listinfo/python-list
>
Quite simple, use xrange (generates the numbers on the fly) rather
than range (creates the list, and gets the numbers from there):
for i in range(sys.maxint):
if i % 100 == 0:
print i
However, I think that the better Python way would be to use a generator:
def infinite_numbergenerator():
n = 0
while True:
yield n
n += 1
for i in infinite_numbergenerator():
...
--
André Engels, andre...@gmail.com
Andre Engels
> for i in range(sys.maxint):
> if i % 100 == 0:
> print i
Grmbl.... cut-and-paste error... I meant of course:
for i in xrange(sys.maxint):
if i % 100 == 0:
print i
--
André Engels, andre...@gmail.com
Rhodri James
> The following code does not run because range() does not accept a big
> number. Is there a way to make the code work. I'm wondering if there
> is a way to write a for-loop in python similar to that of C style.
xrange()
Have you read the documentation at all?
--
Rhodri James *-* Wildebeest Herder to the Masses
Dave Angel
> On Tue, Oct 13, 2009 at 11:55 PM, Andre Engels <andre...@gmail.com> wrote:
>
>
>> for i in range(sys.maxint):
>> print i
>>
>
> Grmbl.... cut-and-paste error... I meant of course:
>
> for i in xrange(sys.maxint):
> print i
>
>
What version of Python gives an error on that code? My 2.6.2 works fine.
If you do need a larger xrange, you can easily write your own. Here's
one called brange(). It'd be a bit more work to make the 3-argument
version.
def brange(limit):
i = 0
while i < limit:
yield i
i += 1
Matt Nordhoff
[snip]
> However, I think that the better Python way would be to use a generator:
>
> def infinite_numbergenerator():
> n = 0
> while True:
> yield n
> n += 1
>
> for i in infinite_numbergenerator():
> ...
That's what itertools.count() is for.
<http://docs.python.org/library/itertools.html#itertools.count>
It'll be faster, too, since it's written in C. Not that it really
matters, but it's always nice.
--
Matt Nordhoff
...Although itertools.count() is limited by sys.maxint (well,
PY_SSIZE_T_MAX), so I guess it isn't useful in this case.
--
Mark Dickinson
<ste...@REMOVE.THIS.cybersource.com.au> wrote:
> On Tue, 13 Oct 2009 16:17:58 -0500, Peng Yu wrote:
> > Hi,
>
> > The following code does not run because range() does not accept a big
> > number.
>
> Incorrect.
>
> >>> range(sys.maxint+2, sys.maxint+5)
>
> [2147483649L, 2147483650L, 2147483651L]
For what it's worth, there *is* a Python oddity lurking
under the surface here, at least for 64-bit. Here's
Python 2.6 on a 64-bit machine:
>>> range(2**31-1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
MemoryError
>>> range(2**31)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: range() result has too many items
The first call tries to allocate a list containing
2**31-1 integers. At 32 bytes per entry (8 for the
list pointer, 24 for the integer itself), that's not
surprising on a machine with < 64 Gb of memory.
The second call, however, doesn't even try to
allocate the memory, but decides that the range can't
be represented. That's not right: this is an LP64
machine, so the size of the list can be represented,
and the start, stop and step values are representable
as C longs (which is what CPython uses internally for
this purpose). On a machine with >64 Gb of memory,
this call should succeed. On an LP64 machine with
<64 Gb of memory, it should produce MemoryError, not
OverflowError.
Mark
David C. Ullrich
<dick...@gmail.com> wrote:
>On Oct 13, 10:39�pm, Steven D'Aprano
><ste...@REMOVE.THIS.cybersource.com.au> wrote:
>> On Tue, 13 Oct 2009 16:17:58 -0500, Peng Yu wrote:
>> > Hi,
>>
>> > The following code does not run because range() does not accept a big
>> > number.
>>
>> Incorrect.
>>
>> >>> range(sys.maxint+2, sys.maxint+5)
>>
>> [2147483649L, 2147483650L, 2147483651L]
>
>For what it's worth, there *is* a Python oddity lurking
>under the surface here, at least for 64-bit. Here's
>Python 2.6 on a 64-bit machine:
Is that a 64-bit Python as well?
Mark Dickinson
> >For what it's worth, there *is* a Python oddity lurking
> >under the surface here, at least for 64-bit. Here's
> >Python 2.6 on a 64-bit machine:
>
> Is that a 64-bit Python as well?
Yes.
Mark
Tim Roberts
>
>That's what itertools.count() is for.
>
><http://docs.python.org/library/itertools.html#itertools.count>
>
>It'll be faster, too, since it's written in C. Not that it really
>matters, but it's always nice.
For being such a simple concept, I've been surprised how many places
itertools.count() has come in handy for me.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.