Re: How to deal with loading corrupted images?

[Don Dwiggins]

Ratko Jagodic wrote:
> Well, try/except is not throwing anything... That's the problem.
> There really is no code snipped. It's just that one line, for example
> im = wx.Image("corrupted_image.jpg")
>
> If the image is corrupted all I get is "Abort" and the program quits.

One possibility, if you don't get a better answer, is to run the image
reading code in a separate subprocess, monitored by the server process.
When the subprocess dies, the server process will start up another.
Use some form of IPC to communicate between the two.

--
Don Dwiggins
Advanced Publishing Technology

[Ratko Jagodic]

I tried:
im = wx.EmptyImage(w,h)
im.LoadFile("corrupted_image.png")

and it aborts the same way. Sometimes I get "Fatal Error: IDAT: CRC error" and then "Abort".
I tried this on a Mac and Linux and I get the same results.

Also, reading the file normally works just fine (without throwing any errors):
f = open("corrupted_file.png", "rb")
data = f.read()
print len(data)
f.close()

On 3/31/06, Michael Hipp <Mic...@hipp.com> wrote:

Does the behavior change if you just create the image empty and then
load it as a second step using im.LoadFile("foo")?

If it's dying without throwing an exception on a corrupt image, that
seems like a bug.

Michael

Ratko Jagodic wrote:
> Well, try/except is not throwing anything... That's the problem.
> There really is no code snipped. It's just that one line, for example
> im = wx.Image("corrupted_image.jpg")
>
> If the image is corrupted all I get is "Abort" and the program quits.

> You can even try it in the interpreter.
> For example a corrupted image would happen after an incomplete transfer.
>
> Ratko
>
>
> On 3/31/06, *Michael Hipp* <Mic...@hipp.com <mailto:Mic...@hipp.com>>
> wrote:
>
>     Can you wrap it in a try/except block?
>
>     What exception is it throwing?
>
>     Might want to post a code snippet.
>
>     Michael
>
>     Ratko Jagodic wrote:
>      > I have a little image server that provides images to the clients
>     over
>      > the network.
>      > Everything works great unless it tries to read an image that is
>      > corrupted (with wx.Image("filename")). In that case, my server just
>      > exits abruptly with an "Abort" or "Fatal Error". Try, except doesnt
>      > catch anything.
>      > Is there any way to register that and handle it so that it
>     doesn't crash
>      > the whole program?
>      > If not, how is this usually handled since I am sure many have
>      > experienced the same problem?
>      > Thanks,
>      >
>      > Ratko
>      >
>
>
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail:
>     wxPython-user...@lists.wxwidgets.org
>     <mailto:wxPython-user...@lists.wxwidgets.org>
>     For additional commands, e-mail:
>     wxPython-...@lists.wxwidgets.org
>     <mailto:wxPython-...@lists.wxwidgets.org >
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-user...@lists.wxwidgets.org
For additional commands, e-mail: wxPython-...@lists.wxwidgets.org

[Christopher Barker]

Ratko Jagodic wrote:
> try:
> f = open("corrupted_image.png", "rb")
> data = f.read()
> f.close()
> stream=wx.InputStream(cStringIO.StringIO(data))
> im = wx.ImageFromStream(stream)
> print "Image size = ", (im.GetWidth(), im.GetHeight())
> except:
> print "exception...."

I'm not sure that would work anyway, I think wx.ImageFromStream only
takes raw rgb image data, not PNG-compressed data. Does it work with a
good PNG?

This sure looks like a wxImage bug. I imagine Robin will be able to make
a ruling on that.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Michael Hipp]

Getting pretty deep into hackarounds instead of actual solutions, but
could you verify its integrity with this:

http://docs.python.org/lib/module-imghdr.html

Before loading the corrupt file. If it works (and I've not tried it) it
would at least provide a way to trap the error.

Michael

Ratko Jagodic wrote:
> Nope, same thing.
>
> try:
> f = open("corrupted_image.png", "rb")
> data = f.read()

> f.close()
> stream=wx.InputStream(cStringIO.StringIO(data))
> im = wx.ImageFromStream(stream)
> print "Image size = ", (im.GetWidth(), im.GetHeight())
> except:
> print "exception...."
>
>
>
>

> On 3/31/06, *Michael Hipp* <Mic...@hipp.com <mailto:Mic...@hipp.com>>
> wrote:
>

> What if you then load that 'data' into wx.Image?

>
> Michael
>
> Ratko Jagodic wrote:
> > I tried:
> > im = wx.EmptyImage(w,h)
> > im.LoadFile("corrupted_image.png")
> >
> > and it aborts the same way. Sometimes I get "Fatal Error: IDAT: CRC
> > error" and then "Abort".
> > I tried this on a Mac and Linux and I get the same results.
> >
> > Also, reading the file normally works just fine (without throwing any
> > errors):
> > f = open("corrupted_file.png", "rb")
> > data = f.read()
> > print len(data)
> > f.close()
> >
> >
> >

> > On 3/31/06, *Michael Hipp* < Mic...@hipp.com

> <mailto:Mic...@hipp.com> <mailto:Mic...@hipp.com
> <mailto:Mic...@hipp.com>>>

> > wrote:
> >
> > Does the behavior change if you just create the image empty
> and then
> > load it as a second step using im.LoadFile("foo")?
> >
> > If it's dying without throwing an exception on a corrupt
> image, that
> > seems like a bug.
> >
> > Michael
> >
> > Ratko Jagodic wrote:
> > > Well, try/except is not throwing anything... That's the
> problem.
> > > There really is no code snipped. It's just that one line,
> for example
> > > im = wx.Image("corrupted_image.jpg")
> > >
> > > If the image is corrupted all I get is "Abort" and the
> program quits.
> > > You can even try it in the interpreter.
> > > For example a corrupted image would happen after an
> incomplete
> > transfer.
> > >
> > > Ratko
> > >
> > >
> > > On 3/31/06, *Michael Hipp* <Mic...@hipp.com
> <mailto:Mic...@hipp.com>

> > <mailto: Mic...@hipp.com <mailto:Mic...@hipp.com>>
> <mailto:Mic...@hipp.com <mailto:Mic...@hipp.com>
> > <mailto:Mic...@hipp.com <mailto:Mic...@hipp.com>>>>

> > <mailto:wxPython-user...@lists.wxwidgets.org
> <mailto:wxPython-user...@lists.wxwidgets.org>>
> > > <mailto:wxPython-user...@lists.wxwidgets.org
> <mailto:wxPython-user...@lists.wxwidgets.org>
> > <mailto: wxPython-user...@lists.wxwidgets.org

> <mailto:wxPython-user...@lists.wxwidgets.org>>>
> > > For additional commands, e-mail:
> > > wxPython-...@lists.wxwidgets.org
> <mailto:wxPython-...@lists.wxwidgets.org>

> > <mailto:wxPython-...@lists.wxwidgets.org
> <mailto:wxPython-...@lists.wxwidgets.org>>
> > > <mailto: wxPython-...@lists.wxwidgets.org
> <mailto:wxPython-...@lists.wxwidgets.org>
> > <mailto:wxPython-...@lists.wxwidgets.org

> <mailto:wxPython-...@lists.wxwidgets.org>>>
> > >
> > >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > wxPython-user...@lists.wxwidgets.org

> <mailto:wxPython-user...@lists.wxwidgets.org>
> > <mailto:wxPython-user...@lists.wxwidgets.org

> <mailto:wxPython-user...@lists.wxwidgets.org>>
> > For additional commands, e-mail:
> > wxPython-...@lists.wxwidgets.org
> <mailto:wxPython-...@lists.wxwidgets.org>

> > <mailto:wxPython-...@lists.wxwidgets.org

[Robin Dunn]

Ratko Jagodic wrote:
> Same error while the imghdr doesn't report any.

Presumably that means that the header is intact, and it is not until
later in the file that the corruption happens.

>
> try:
> print "Image type: ", imghdr.what("corrupted_image.png")
> im = wx.Image("corrupted_image.png")

> print "Image size = ", (im.GetWidth(), im.GetHeight())
> except:

> print "exception!!!"
>
> OUTPUT:
> Image type: png

> Fatal Error: IDAT: CRC error

> Abort
>
>
> So, there is no way of checking/catching/recovering/dealing with Fatal
> Errors? It's certain death?
> BTW, I tried opening the file with ImageMagick's "display" and with GIMP
> and they both report a corrupt image but they dont crash... which is
> exactly what I would need with wxPython.

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

[Ratko Jagodic]

Yes, you are right. It does accept raw RGB data. I just copied and pasted that from another one of my apps without looking at the context it's in. But still, it failed... With a good image it reports "Warning: No handler found for image type." and it reports the size as (0,0). With the corrupted image it never even gets there, it just reports the usual:

Fatal Error: IDAT: CRC error

Abort

[Josiah Carlson]

"Ratko Jagodic" <rjag...@gmail.com> wrote:
> > If you're not creating a GUI wouldn't something like PIL be a better
> choice?
>
> Well, the only reason why I chose wx for this is because it's platform
> independent without compilation. And since everybody already has wxPython
> because of my other apps, they can just run this without any installation
> issues. I understand that PIL would be more efficient but speed is not
> really an issue since everything is running over the network anyway.
> Actually, the only thing that the server needs to do is open an image,
> resize it (make a thumbnail and cache it) and get it's size in pixels. So
> far images of up to 100MB have worked flawlessly and were processed in less
> than 10 secs.
> Again, thanks for the help.

Based on your comments, it would seem that the major potential source
for corruption of your images, if you assume that your server works
correctly; is either a partial image download, or the rare (but possible)
network transfer corruption.

There are methods of dealing with those two issues, namely a better
transfer protocol. If you prefix the data of your image with a packed
representation of the length of the image (import struct;struct.pack('>L',
len(img_data)) ), along with a checksum of the image data (import
sha1;sha1.new(img_data).digest() ), you can verify the data was recieved
correctly quite easily, and save your software from crashing due to a
buggy image.

- Josiah

> On 4/1/06, Robin Dunn <ro...@alldunn.com> wrote:
> >
> > Ratko Jagodic wrote:
> > > OK, more news. Apparently the problem has been fixed in 2.6.3 and now
> > > all the image handlers throw an assertion error. However, the
> > > application keeps on running which is good.
> > >
> > > My question is, how can I catch those assertions if I never even create
> > > wx.App() ?
> >
> > It's actually not an assertion, but a wxLogError message, so it can't
> > currently be caught.
> >
> > > I only use wx.Image from wx for its image saving and loading
> > > capabilities and of course platform independence.
> >
> > If you're not creating a GUI wouldn't something like PIL be a better
> > choice?

> >
> >
> > --
> > Robin Dunn
> > Software Craftsman
> > http://wxPython.org Java give you jitters? Relax with wxPython!
> >
> >

[Ratko Jagodic]

In what sense do you mean that wx is a lot of overhead? Speed wise or memory wise?
Speed wise PIL and wx are almost the same (in a few tests I did wx was less than 5% slower than PIL... just for image resizing and loading which is what I need).

I agree that PIL is by far better suited for image processing but in my case wx works just as well and people that will be using my app already have wxPython so there's no need for them to compile anything (PIL) to get it running.

One nice feature of PIL is that getting the image size in pixels is very fast since it apparently just reads the header while wx needs to read the whole image into memory first. For that I use a piece of code that someone posted on the Py Cookbook for reading image headers. BTW, wouldn't that be a nice standard feature in wx?? Or maybe even more appropriately in python's imghdr module?

Ratko

On 4/3/06, Christopher Barker <Chris....@noaa.gov> wrote:

Ratko Jagodic wrote:

>     I only use wx.Image from wx for its image saving and loading
>     capabilities and of course platform independence.

wx is kind of a lot of overhead for that? Why not PIL? Or on of a few
others I can't think of the names of at the moment.

-Chris



--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-user...@lists.wxwidgets.org