How to iterate the input over a particular size?

1 view
Skip to first unread message

joy99

unread,
Dec 27, 2009, 9:44:17 AM12/27/09
to
Dear Group,

I am encountering a small question.

Suppose, I write the following code,

input_string=raw_input("PRINT A STRING:")
string_to_word=input_string.split()
len_word_list=len(string_to_word)
if len_word_list>9:
rest_words=string_to_word[9:]
len_rest_word=len(rest_words)
if len_rest_word>9:
remaining_words=rest_words[9:]


In this program, I am trying to extract first 9 words from an
indefinitely long string, until it reaches 0.
Am I writing it ok, or should I use while, or lambda?
If any one can suggest.

Hope you are enjoying a nice vacation of Merry Christmas. If any one
is bit free and may guide me up bit.

Wishing you a happy day ahead,
Best Regards,
Subhabrata.

Shawn Milochik

unread,
Dec 27, 2009, 10:14:01 AM12/27/09
to pytho...@python.org
A couple of notes:

Your code is too packed together. That makes it hard to read. Put in some whitespace (and comments, where necessary), and put spaces around your equal signs (x = 23 instead of x=23).

That said, here's something I threw together:

#!/usr/bin/env python

def chop_list(words, size = 9):

"""
Take a list, returning values
from the end of the list, and
the original list minus those values.

>>> chop_list(['test', 'with', 'four', 'words'], 2)
(['test', 'with'], ['four', 'words'])

>>> chop_list(['test', 'with', 'four', 'words'], 3)
(['test'], ['with', 'four', 'words'])

>>> chop_list(['test', 'with', 'four', 'words'], 4)
([], ['test', 'with', 'four', 'words'])


"""

chopped = words[-size:]
old_to_return = len(words) - len(chopped)

return words[:old_to_return], chopped


if __name__ == '__main__':

import doctest
doctest.testmod()

sample_string = '''There are more than nine words here.
I know because I wrote them.'''

word_list = sample_string.split()

while word_list:

word_list, new = chop_list(word_list)
print "Pulled off:\t%s\n\tRemaining: %s" % (new, word_list)

Benjamin Kaplan

unread,
Dec 27, 2009, 10:42:41 AM12/27/09
to pytho...@python.org
> --

You want the first 9 words? string_to_word[:9]
You want the last 9 words? string_to_word[-9:]

If you want the groups of words, use a loop- that's the only way to
get all of them for any length list.

> http://mail.python.org/mailman/listinfo/python-list
>

joy99

unread,
Dec 27, 2009, 4:29:57 PM12/27/09
to
On Dec 27, 8:42 pm, Benjamin Kaplan <benjamin.kap...@case.edu> wrote:
> >http://mail.python.org/mailman/listinfo/python-list- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

Dear Group,
Answers were good. But I am looking for a smarter solution like:

for i[:2] in list:
....

etc. or by doing some looping over loop.
Do not worry I'll work out the answer.

Wishing you a happy day ahead,

Regards,
Subhabrata.

Francesco Bochicchio

unread,
Dec 28, 2009, 3:58:28 AM12/28/09
to
> > >http://mail.python.org/mailman/listinfo/python-list-Hide quoted text -

>
> > - Show quoted text -- Hide quoted text -
>
> > - Show quoted text -
>
> Dear Group,
> Answers were good. But I am looking for a smarter solution like:
>
> for i[:2] in list:
> ....
>
> etc. or by doing some looping over loop.
> Do not worry I'll work out the answer.
>
> Wishing you a happy day ahead,
> Regards,
> Subhabrata.

Not sure I understood your question, but if you need just to plit a
big list in sublists of no more than 9 elements, then you can do
someting like:

def sublists(biglist, n ):
"Splits a big list in sublists of max n elements"
prev_idx = 0; res = []
for idx in xrange(n, len(biglist)+n, n ):
res.append( biglist[prev_idx:idx] )
prev_idx = idx
return res

I would not be surprised if somewhere in python standard library there
is something like this (possibly better), but
could not find anything.

Another solution could be this smarter-looking but less readeable one
liner:

sublists = [ big_list[i:(i+9)] for i in xrange( 0, len
(big_list)+9, 9) if i < len(big_list) ]


P.S : if your biglist is huge (but being typed in I don't think so)
then you better convert the "sublists" function in a
generator.


HTH

Ciao
----
FB

Peter Otten

unread,
Dec 28, 2009, 4:54:55 AM12/28/09
to
Francesco Bochicchio wrote:

> Not sure I understood your question, but if you need just to plit a
> big list in sublists of no more than 9 elements, then you can do
> someting like:
>
> def sublists(biglist, n ):
> "Splits a big list in sublists of max n elements"
> prev_idx = 0; res = []
> for idx in xrange(n, len(biglist)+n, n ):
> res.append( biglist[prev_idx:idx] )
> prev_idx = idx
> return res
>
> I would not be surprised if somewhere in python standard library there
> is something like this (possibly better), but
> could not find anything.

A lazy alternative:

>>> from itertools import islice
>>> def chunks(items, n):
... items = iter(items)
... for first in items:
... chunk = [first]
... chunk.extend(islice(items, n-1))
... yield chunk
...
>>> list(chunks(range(10), 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

One with lazy chunks:

>>> from itertools import chain, islice
>>> def chunks(items, n):
... items = iter(items)
... for first in items:
... yield chain((first,), islice(items, n-1))
...
>>> [list(chunk) for chunk in chunks(range(10), 3)]
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

Prior art by Duncan Booth:
http://mail.python.org/pipermail/python-list/2007-November/517993.html

Peter

Vlastimil Brom

unread,
Dec 28, 2009, 7:28:32 AM12/28/09
to pytho...@python.org
2009/12/27 joy99 <subhakol...@gmail.com>:

> Dear Group,
> Answers were good. But I am looking for a smarter solution like:
>
> for i[:2] in list:
> ....
>
> etc. or by doing some looping over loop.
> Do not worry I'll work out the answer.
>
> Wishing you a happy day ahead,
> Regards,
> Subhabrata.

> --
> http://mail.python.org/mailman/listinfo/python-list
>
Hi,
maybe just something like:

>>> input_list = range(10)
>>> n = 3
>>> [input_list[i:i+n] for i in range(0, len(input_list), n)]


[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
>>>

?
(possibly using xrange and generator expression instead of the list
comprehension)

vbr

Lie Ryan

unread,
Dec 28, 2009, 12:15:16 PM12/28/09
to
On 12/28/2009 8:54 PM, Peter Otten wrote:

> Francesco Bochicchio wrote:
> One with lazy chunks:
>
>>>> from itertools import chain, islice
>>>> def chunks(items, n):
> .... items = iter(items)
> .... for first in items:
> .... yield chain((first,), islice(items, n-1))
> ....

>>>> [list(chunk) for chunk in chunks(range(10), 3)]
> [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]


better:
import itertools
def chunks(items, n):
ctr = (x // n for x in itertools.count())
return itertools.groupby(items, lambda _: next(ctr))

input_list = range(10)
list(list(x[1]) for x in chunks(input_list, 3))
# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

joy99

unread,
Dec 28, 2009, 12:24:08 PM12/28/09
to
On Dec 28, 5:28 pm, Vlastimil Brom <vlastimil.b...@gmail.com> wrote:
> 2009/12/27 joy99 <subhakolkata1...@gmail.com>:

vbr,
Just Brilliant.
I manipulated it bit but I got the key, but thanks to all for giving
me lots of solutions and lots of insights over Python. Everytime I
visit this room I learn so much. Thank you all.

Wishing a Great Day ahead,

Best Regards,
Subhabrata.

Peter Otten

unread,
Dec 28, 2009, 1:10:29 PM12/28/09
to
Lie Ryan wrote:

I disagree. It may be of little practical relevance that this fails after
about sys.maxint items (Python 2.5) or at least slows down (Python 2.6) when
Python long integer arithmetic kicks in, but I still feel uneasy about it.
If I were to use groupby() I'd probably code the "counter" as

ctr = cycle([False]*n + [True]*n)

Peter

John Posner

unread,
Dec 29, 2009, 12:49:50 AM12/29/09
to
On Sun, 27 Dec 2009 09:44:17 -0500, joy99 <subhakol...@gmail.com>
wrote:

> Dear Group,
>
> I am encountering a small question.
>
> Suppose, I write the following code,
>
> input_string=raw_input("PRINT A STRING:")
> string_to_word=input_string.split()
> len_word_list=len(string_to_word)
> if len_word_list>9:
> rest_words=string_to_word[9:]
> len_rest_word=len(rest_words)
> if len_rest_word>9:
> remaining_words=rest_words[9:]
>

Here's an issue that has not, I think, been addressed in this thread. The
OP's problem is:

1. Start with an indefinitely long string.

2. Convert the string to a list, splitting on whitespace.

3. Repeatedly return subslices of the list, until the list is exhausted.

This thread has presented one-chunk-at-a-time (e.g. generator/itertools)
approaches to Step #3, but what about Step #2? I've looked in the Python
documentation, and I've done some Googling, but I haven't found a
generator version of the string function split(). Am I missing something?

Tx,
John


Ben Finney

unread,
Dec 29, 2009, 1:34:39 AM12/29/09
to
"John Posner" <jjpo...@optimum.net> writes:

> I haven't found a generator version of the string function split(). Am
> I missing something?

To my knowledge there isn't one. Both Python 2 and Python 3 document
‘str.split’ as returning a list.

--
\ “An idea isn't responsible for the people who believe in it.” |
`\ —Donald Robert Perry Marquis |
_o__) |
Ben Finney

Steven D'Aprano

unread,
Dec 29, 2009, 1:35:41 AM12/29/09
to

"Indefinitely long" doesn't mean you can't use split.

But if you want a lazy splitter, here's a version which should do what
you want:


def lazy_split(text):
accumulator = []
for c in text:
if c in string.whitespace:
if accumulator:
yield ''.join(accumulator)
accumulator = []
else:
accumulator.append(c)
if accumulator:
yield ''.join(accumulator)


Other alternatives are to use a regex to find runs of whitespace
characters, then yield everything else; or to use the itertools.groupby
function.


--
Steven

John Posner

unread,
Dec 29, 2009, 12:54:09 PM12/29/09
to Steven D'Aprano

Thanks -- I coded a similar generator, too. Yours handles the corner cases
and end-of-string more elegantly, so I won't share mine. :-)

>
> Other alternatives are to use a regex to find runs of whitespace
> characters, then yield everything else; or to use the itertools.groupby
> function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,
get a life, buddy!):

def groupby_split(text):
whitespace_grouper = itertools.groupby(
text,
lambda(c): c not in string.whitespace)
for boolval, group_iter in itertools.ifilter(
lambda pair: pair[0] == True,
whitespace_grouper):
yield "".join(group_iter)

Tx,
John

John Posner

unread,
Dec 29, 2009, 12:54:37 PM12/29/09
to Steven D'Aprano
On Tue, 29 Dec 2009 01:35:41 -0500, Steven D'Aprano
<ste...@remove.this.cybersource.com.au> wrote:

Thanks -- I coded a similar generator, too. Yours handles the corner cases


and end-of-string more elegantly, so I won't share mine. :-)

>


> Other alternatives are to use a regex to find runs of whitespace
> characters, then yield everything else; or to use the itertools.groupby
> function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,

John Posner

unread,
Dec 29, 2009, 12:57:51 PM12/29/09
to Steven D'Aprano
On Tue, 29 Dec 2009 01:35:41 -0500, Steven D'Aprano
<ste...@remove.this.cybersource.com.au> wrote:

Thanks -- I coded a similar generator, too. Yours handles the corner cases


and end-of-string more elegantly, so I won't share mine. :-)

>


> Other alternatives are to use a regex to find runs of whitespace
> characters, then yield everything else; or to use the itertools.groupby
> function.

Yup, that approach occurred to me as I was falling asleep last night (OMG,

Aahz

unread,
Jan 13, 2010, 3:37:16 PM1/13/10
to
In article <46a5e979-7a87-49ee...@e37g2000yqn.googlegroups.com>,

joy99 <subhakol...@gmail.com> wrote:
>
>input_string=raw_input("PRINT A STRING:")
>string_to_word=input_string.split()
>len_word_list=len(string_to_word)
>if len_word_list>9:
> rest_words=string_to_word[9:]
> len_rest_word=len(rest_words)
> if len_rest_word>9:
> remaining_words=rest_words[9:]

input_string.split(None, 9) does something like what you want, but it
does require recopying the tail each iteration. Overall, if you have
enough RAM and you don't need to preserve input_string after splitting,
your approach probably is the one I'd use.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur." --Red Adair

Reply all
Reply to author
Forward
0 new messages