How to load an image from jpeg encoded string

[Sravan Kumar Reddy Javaji]

Hello Everyone,

I have a string which contains image data in jpeg encoded format. 

gm.Image():fromString() function expects normal string, so when I provided jpeg encoded string it is giving empty image (i.e., image size of 0)

im= gm.Image()

im:fromString(data)    -- data contains jpeg encoded string 

print(#im)    -- displays 0

image:load() expects a filename. So, probably we may have to write the jpeg encoded string data into temporary file and this file has to be provided as input to the image:load(). But this step effects run time performance a lot. I have around 50 K images.

In caffe there is image.open() function and StringIO. So, using the below code it is possible to convert the string to image

>>>from StringIO import StringIO

>>>from PIL import Image

>>>image_file = StringIO(open("test.jpg",'rb').read())

>>>im = Image.open(image_file)

>>>print im.size, im.mode

(2121, 3508) RGB

stringio is also present in Lua Penlight, but the problem is in Torch there is no image:open() function, thus there is no way to provide already opened image file handler as input to image routines, all the existing image routines expects filename as parameter.

Currently I am looking into gd.createFromJpegStr(string) function of Lua-GD: https://ittner.github.io/lua-gd/manual.html

Could someone please let me know if there is any other way. Thanks for your time and help.

-

Regards,

Sravan

[Jonathan Tompson]

Hey,

So I wrote a function in image a while back that will decode a jpeg from a ByteTensor.  Is is probably what you're looking for.  So you would read in the image file as a binary and then call decode when you actually want the full image.  The function is 'decompressJPG' and it looks like it's undocumented.

I think when I wrote it image didn't have much documentation...

Jonathan

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at http://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Jonathan Tompson]

Here is the associated PR:

https://github.com/torch/image/pull/30

[Sravan Kumar Reddy Javaji]

Jonathan, You rock!

image.decompressJPG() function perfectly works for me. Thank you very much for your help :) 

-

Regards,

Sravan

On Wednesday, March 4, 2015 at 1:24:24 PM UTC-8, Jonathan Tompson wrote:

Here is the associated PR:

https://github.com/torch/image/pull/30

On Wed, Mar 4, 2015 at 4:22 PM, Jonathan Tompson <jonatha...@gmail.com> wrote:

Hey,

So I wrote a function in image a while back that will decode a jpeg from a ByteTensor.  Is is probably what you're looking for.  So you would read in the image file as a binary and then call decode when you actually want the full image.  The function is 'decompressJPG' and it looks like it's undocumented.

I think when I wrote it image didn't have much documentation...

Jonathan

[Sravan Kumar Reddy Javaji]

Hello Jonathan,

I am using image.decompressJPG on around 100 images, then I am getting the below error,

*** Error in `/home/ubuntu/torch/install/bin/luajit': free(): invalid next size (fast): 0x00007f32439eff00 ***
Aborted (core dumped)

I am not sure is this is because of  "image.decompressJPG()" function as I am doing some other steps like reading images randomly and preprocessing the image.

Just in a quick testing, I commented "image.decompressJPG()" function and ran the same code on normal images (i.e., decoded images). Then job ran fine.

I believe that there is a memory leak issue in some function. At present my code is huge, so I am not able to post it here. I will try to recreate the scenario using small code and will post here for your review.

Thanks for your time.

-
Regards,
Sravan

 

[Jonathan Tompson]

Ahh...  It does sound like a memory bug...  Interesting.

It looks like Ronan submitted a few bug fixes specifically to fix leaks.  Are you using the latest commit?

--

You received this message because you are subscribed to the Google Groups "torch7" group.

[Sravan Kumar Reddy Javaji]

Oh. I am not using the latest commit. I will test with latest commit and let you know the result. Thanks.

-
Regards,
Sravan

On Saturday, March 28, 2015 at 1:33:16 PM UTC-7, Jonathan Tompson wrote:

Ahh...  It does sound like a memory bug...  Interesting.

It looks like Ronan submitted a few bug fixes specifically to fix leaks.  Are you using the latest commit?

[soumith]

I've been using it, seem to see no memory explosion. I've been using latest commit.

--S

[Sravan Kumar Reddy Javaji]

Yes, Now I tested only the function which decodes the images. Its working fine.

Basically I am reading images randomly from lmdb and decoding those images. Then I am facing this memory issue. Today I will put the code in github and let you know. Please check it once. Appreciate your help :)

-
Regards,
Sravan

On Monday, March 30, 2015 at 10:26:55 AM UTC-7, smth chntla wrote:

I've been using it, seem to see no memory explosion. I've been using latest commit.

--S

[Sravan Kumar Reddy Javaji]

Hello Jonathan/Soumith,

Is there any method to load an image from png encoded string like decompressJPG(). 

Thanks for your help :)

-

Regards,

Sravan

[Sravan Kumar Reddy Javaji]

I started working on the changes for decompressPNG(). Now I found that Nicholas completed this :)

Thanks Nicholas :)

-

Regards,

Sravan