Nested For loop not running full

[inshu chauhan]

Hello everyone, 

I have this part of my code where I am trying to traverse over an image by running a for loop for both x and y co-ordinate axis. But the loop is terminating by just reading first pixel. Can think of a reason why this is happening ?

The code is: 

for sy in xrange(0, segimage.height):

    for sx in xrange(0, segimage.width):

            if segimage[sy,sx] == (0.0, 0.0, 0.0):

                continue

            else:

                seg_color = segimage[sy,sx]

                blue = int(seg_color[0])

                green = int(seg_color[1])

                red = int(seg_color[2])

                reg_num = blue + 256 * green + 65536 * red

                for l in f:

                    sp = l.split(",")

                    if len(sp) == 14:

                        print sy, sx  # for checking which pixel its reading currently

                        print reg_num, sp[0]  # for checking whats happening  

                        if reg_num == int(sp[0].strip()):

                            print reg_num, sp[0].strip() # for checking whats happening

                            classification = int(sp[13].strip())

                            

The inside "for loop" is for reading a csv format file from which I am extracting some information. 

Thanks in Advance for your suggestions

                         

[Peter Otten]

My crystal ball says that the 'for sy...' and 'for sx...' loops are running
to completion, but you don't get the coordinates printed because you put
them into the 'for l in f' loop which will only run once.

The quick and dirty fix is to replace

f = open(...)

in the code you are not showing with

f == list(open(...))

The reasonable thing to do is of course to move the preprocessing (e.g. csv-
parsing) out of the sy and sx loops.

[Terry Jan Reedy]

On 4/26/2013 4:48 AM, inshu chauhan wrote:
> Hello everyone,

>
> I have this part of my code where I am trying to traverse over an image
> by running a for loop for both x and y co-ordinate axis. But the loop is
> terminating by just reading first pixel. Can think of a reason why this
> is happening ?

*A* reason could be that segimage.height and .width are both 1. I would
print them out to see what they are.

>
> The code is:
> for sy in xrange(0, segimage.height):
> for sx in xrange(0, segimage.width):
> if segimage[sy,sx] == (0.0, 0.0, 0.0):
> continue
> else:
> seg_color = segimage[sy,sx]
> blue = int(seg_color[0])
> green = int(seg_color[1])
> red = int(seg_color[2])
> reg_num = blue + 256 * green + 65536 * red
> for l in f:
> sp = l.split(",")
> if len(sp) == 14:
> print sy, sx # for checking which pixel its
> reading currently
> print reg_num, sp[0] # for checking whats
> happening
> if reg_num == int(sp[0].strip()):
> print reg_num, sp[0].strip() # for checking
> whats happening
> classification = int(sp[13].strip())
>
> The inside "for loop" is for reading a csv format file from which I am
> extracting some information.
>

[inshu chauhan]

On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pe...@web.de> wrote:

inshu chauhan wrote:

> I have this part of my code where I am trying to traverse over an image by
> running a for loop for both x and y co-ordinate axis. But the loop is
> terminating by just reading first pixel. Can think of a reason why this is
> happening ?
>

> The code is:
> for sy in xrange(0, segimage.height):
>     for sx in xrange(0, segimage.width):
>             if segimage[sy,sx] == (0.0, 0.0, 0.0):
>                 continue
>             else:
>                 seg_color = segimage[sy,sx]
>                 blue = int(seg_color[0])
>                 green = int(seg_color[1])
>                 red = int(seg_color[2])
>                 reg_num = blue + 256 * green + 65536 * red
>                 for l in f:
>                     sp = l.split(",")
>                     if len(sp) == 14:
>                         print sy, sx  # for checking which pixel its
> reading currently
>                         print reg_num, sp[0]  # for checking whats
> happening
>                         if reg_num == int(sp[0].strip()):
>                             print reg_num, sp[0].strip() # for checking
> whats happening
>                             classification = int(sp[13].strip())
>
>
> The inside "for loop" is for reading a csv format file from which I am
> extracting some information.

My crystal ball says that the 'for sy...' and 'for sx...' loops are running
to completion, but you don't get the coordinates printed because you put
them into the 'for l in f' loop which will only run once.

Is there any means by which I can run this 'For l in f' loop again and again ?  

The quick and dirty fix is to replace

f = open(...)

in the code you are not showing with

f == list(open(...))

 

f is just a text file(csv format).. so why list ??

 

The reasonable thing to do is of course to move the preprocessing (e.g. csv-
parsing) out of the sy and sx loops.

I did this but again then what I intend to do is not really happening, For every pixel I read,  I want to traverse the full file, so that the information I am taking from pixel have to match in one of the line in the file. Can this be done by modifying my code ? or something new has to be devised ?

--
http://mail.python.org/mailman/listinfo/python-list

[Oscar Benjamin]

On 26 April 2013 10:36, inshu chauhan <insid...@gmail.com> wrote:

>
> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pet...@web.de> wrote:
>>
>> My crystal ball says that the 'for sy...' and 'for sx...' loops are
>> running
>> to completion, but you don't get the coordinates printed because you put
>> them into the 'for l in f' loop which will only run once.
>
> Is there any means by which I can run this 'For l in f' loop again and again
> ?
>>
>> The quick and dirty fix is to replace
>>
>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>
> f is just a text file(csv format).. so why list ??

So that you can run the for l in f loop again and again. You can loop
over a list as many times as you like but only once over a file
(unless you reset the file pointer).


Oscar

[Chris Angelico]

On Fri, Apr 26, 2013 at 7:36 PM, inshu chauhan <insid...@gmail.com> wrote:
>
> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pet...@web.de> wrote:
>>
>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>
> f is just a text file(csv format).. so why list ??

(That should be =, not ==)

Instead of having an open file object, you would instead have a list
of the lines in the file. That can be iterated over more than once.

>> The reasonable thing to do is of course to move the preprocessing (e.g.
>> csv-
>> parsing) out of the sy and sx loops.
>
>
> I did this but again then what I intend to do is not really happening, For
> every pixel I read, I want to traverse the full file, so that the
> information I am taking from pixel have to match in one of the line in the
> file. Can this be done by modifying my code ? or something new has to be
> devised ?

How large is the file? There are two easy solutions:

1) Open and close the file every time you touch a pixel
2) Open the file once, read it all into memory, and then iterate over
the in-memory copy every pixel

If your file is insanely large then the first option may be better,
but for anything less than five yottabytes, go with the second. (Okay,
I may be exaggerating slightly... let's say anything less than half
your RAM. So if you have 10YB of memory, then I wasn't exaggerating.)
That's why Peter suggested creating a list; you iterate over the list.
Another way to do it is to parse the file once and retain a more
efficient and useful structured form of the data... which is the other
thing Peter suggested ("move the preprocessing (e.g. csv-
parsing) out of the sy and sx loops").

So, yeah. Listen to Peter Otten, he knows what he's talking about :)

ChrisA

[Peter Otten]

inshu chauhan wrote:

> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pet...@web.de> wrote:
>

>> My crystal ball says that the 'for sy...' and 'for sx...' loops are
>> running to completion, but you don't get the coordinates printed because
>> you put them into the 'for l in f' loop which will only run once.
>>
>
> Is there any means by which I can run this 'For l in f' loop again and
> again ?
>
>>
>> The quick and dirty fix is to replace
>>

>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>>
>
> f is just a text file(csv format).. so why list ??

Can you figure it out yourself from the following?

>>> f = open("tmp.data")
>>> for i in range(3):
... for line in f: print repr(line)
...
'alpha\n'
'beta\n'
'gamma\n'
>>> f = list(open("tmp.data"))
>>> for i in range(3):
... for line in f: print repr(line)
...
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'

>> The reasonable thing to do is of course to move the preprocessing (e.g.
>> csv-
>> parsing) out of the sy and sx loops.
>>
>
> I did this but again then what I intend to do is not really happening, For
> every pixel I read, I want to traverse the full file, so that the
> information I am taking from pixel have to match in one of the line in the
> file. Can this be done by modifying my code ? or something new has to be
> devised ?

I think I have already answered this, but here's another alternative:

>>> f = open("tmp.data")
>>> for i in range(3):
... f.seek(0)
... for line in f: print(repr(line))
...
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'

[Jens Thoms Toerring]

Are you sure that the loop is only run once? In that case the most
likely thing is that the image consists of only a single pixel
(or all except the first one are black, then it might also look
as if the loop would be run only once;-)

But what looks strange is the innermost loop. You never tell what
exactly 'f' is but I would tend to assume that it is a file object
for your CSV file, which you opened somewhere before. And now you
read it in completely when dealing with the very first pixel of
your image. Afterwards, when dealing with the other pixels of the
image, there's nothing left to be read in, so the inner loop won't
be run again, making it appear as if the outer loops would only be
run once.

If my assumptions are correct and you want to read in the file
again and again for each pixel then you should either open it
again and again for each pixel or, probably better, reset the
file object so that it "points" back to the start of the file
before the start of the innermost loop, using the seek() method
- a simple "f.seek(0)" should do the job (assuming that this is
a normal file, i.e. one that can be "rewound" and not e.g. a re-
directed pipe).

An even better solution (if you have enough memory) might be to
read in the whole file into a list and iterate over that instead
of the file itself. And better than that might be to build a
dictionary of values in the file that you can use later on, so
you don't have to run over the whole file again and again:

d = { }
for l in f :
sp = split( l, ',' )
if len( sp ) == 14 :
d[ int( sp[ 0 ].strip( ) ) ] = int( sp[ 13 ].strip( ) )

Then you can later check directly if some color value (what
you have named 'reg_num') is in the file by using

if reg_num in d :

and the corresponding value from the file (what you assign
to 'classification') is simply the value of the dictionary
for the key given by 'reg_num'. i.e.

classification = d[ reg_num ]

Regards, Jens
--
\ Jens Thoms Toerring ___ j...@toerring.de
\__________________________ http://toerring.de

[inshu chauhan]

, 2013 at 3:15 PM, Chris Angelico <ros...@gmail.com> wrote:

On Fri, Apr 26, 2013 at 7:36 PM, inshu chauhan <insid...@gmail.com> wrote:
>
> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pet...@web.de> wrote:
>>

>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>
> f is just a text file(csv format).. so why list ??

(That should be =, not ==)

Instead of having an open file object, you would instead have a list
of the lines in the file. That can be iterated over more than once.

>> The reasonable thing to do is of course to move the preprocessing (e.g.
>> csv-
>> parsing) out of the sy and sx loops.
>
>
> I did this but again then what I intend to do is not really happening, For
> every pixel I read,  I want to traverse the full file, so that the
> information I am taking from pixel have to match in one of the line in the
> file. Can this be done by modifying my code ? or something new has to be
> devised ?

How large is the file? There are two easy solutions:

1) Open and close the file every time you touch a pixel
2) Open the file once, read it all into memory, and then iterate over
the in-memory copy every pixel

If your file is insanely large then the first option may be better,
but for anything less than five yottabytes, go with the second. (Okay,
I may be exaggerating slightly... let's say anything less than half
your RAM. So if you have 10YB of memory, then I wasn't exaggerating.)
That's why Peter suggested creating a list; you iterate over the list.
Another way to do it is to parse the file once and retain a more
efficient and useful structured form of the data... which is the other

thing Peter suggested ("move the preprocessing (e.g. csv-

parsing) out of the sy and sx loops").

So, yeah. Listen to Peter Otten, he knows what he's talking about :)

ChrisA

Yes I am trying Peter's way and my file is just 500 KB .. :)