A Simple Question - How do I convert between Bitmap and Mat?

[Noam]

I have an Android Bitmap in my code and I would like to run the
cvCanny method on it. However, it needs to be in a Mat first. How do I
convert the data to Mat, and how do I convert it back to Bitmap when
I'm done?

I tried using the Utils.bitmapToMat and matToBitmap methods, but they
don't seem to work. Can someone provide some simple sample code?
Here's an example of what I attempted:

Mat mcv = org.opencv.android.Utils.bitmapToMat(bm);
org.opencv.android.Utils.matToBitmap(mcv, bm2);

[Aaron Brown]

Noam,

The matToBitmap and BitmapToMat methods only support RGBA_8888
pixel format. Any other format and they return a 0x0 Mat object.

Unfortunately, I'm running into the same problem you are, because
I'm trying to work with images captured from the Android camera and
Android doesn't support RGBA_8888 for camera capture. :-/

-AMB

[Noam]

Is there a way to convert the format after the picture is taken?

[Aaron Brown]

I'm working on a way right now. My idea is to take the picture in
RGB565 and then handle the 16-bit to 32-bit conversion myself (packing
the RGB values into their own bytes and then adding a full-opacity
alpha channel.) It looks like Sample 0 in the Android OpenCV samples
does something like this, so I'm going to borrow heavily from that
code.

If I find anything that works, I'll be sure to post the code here.

-Aaron

[Aaron Brown]

I was wrong about what the code in Sample0 does. It looks like it
converts YUV to RGB.

Still, I'm going to try and figure out a way to do the conversion
myself. I'll let you know if I have any luck.

-Aaron

[Noam]

I decided to hack my way around this by reading/writing to the disk.
It's not a great solution, but I only need to do the conversion a
couple times in the program so the loss of speed is negligible for me.
However, if you find a better solution I would love to see it.

[Aaron Brown]

So I found a surprisingly simple method that appears to work. The
method below converts JPEGs, as captured by the camera, and converts
them to RGB888. I've run these through BitmapToMat and back again, as
well as successfully running them through a CascadeClassifier:

/** Takes a JPEG captured by the device camera and converts it to
RGB888 format */
private Bitmap JPEGtoRGB888(Bitmap img)
{
int numPixels = img.getWidth()* img.getHeight();
int[] pixels = new int[numPixels];

//Get JPEG pixels. Each int is the color values for one
pixel.
img.getPixels(pixels, 0, img.getWidth(), 0, 0, img.getWidth(),
img.getHeight());

//Create a Bitmap of the appropriate format.
Bitmap result = Bitmap.createBitmap(img.getWidth(),
img.getHeight(), Config.ARGB_8888);

//Set RGB pixels.
result.setPixels(pixels, 0, result.getWidth(), 0, 0,
result.getWidth(), result.getHeight());
return result;
}

Usage:

Be sure that your camera format is set to JPEG:

Camera.Parameters parameters = myCamera.getParameters();
//RGB888 not natively supported by the camera
parameters.setPictureFormat(ImageFormat.JPEG);
myCamera.setParameters(parameters);

In your JPEG camera callback:

PictureCallback handleJpeg = new PictureCallback() {
public void onPictureTaken(byte[] _data, Camera _camera) {
Bitmap bmp=BitmapFactory.decodeByteArray(_data,0,_data.length);
bmp = JPEGtoRGB888(bmp);
...
}
}


Like I said, this seems to work for me. It's matching just fine
through CascadeClassifiers can be run through the Mat conversion
methods.

For some reason I'm having trouble marking up the resulting Mats with
Core.rectangle, but I'm 80% confident that that's unrelated.

I hope this helps!

-Aaron

[Aaron Brown]

Just updating to say that the reason it wasn't marking the detected
objects? I'd commented out the line that refreshes my displayed
image. *Headdesk*

This method seems to totally work. :-)

Hope it helps!

-Aaron

[Noam]

What about the reverse -- going from a Mat to a Bitmap? The built-in
utility does not seem to work for that.

[Maurilio Silva]

Hi Noam,

I'm using this way:

public Bitmap myMethod(...)
{
...
        Imgproc.cvtColor(imageMat, imageOut, Imgproc.COLOR_BGR2BGRA, 4); // Use the correct color conversion to you case

        Bitmap bmp = Bitmap.createBitmap(imageOut.cols(), imageOut.rows(), Bitmap.Config.ARGB_8888);

        if (android.MatToBitmap(imageOut, bmp))
            return bmp;

        bmp.recycle();

        return null;
}
...



--
Best regards,

Maurílio Silva

-----------------------------------------------------------------------
Master's candidate in Computer Science at UFCG
Mobile/Game Developer
Forum Nokia Champion

MeX (Mobile eXperience Group) -  We Will MeX You!
-----------------------------------------------------------------------

Join us at  http://community.forum.nokia.com

[strothm]

Hey all,

a pretty awesome project you guys have. This is my first post here. I
faced the same problem during June. The problem seems to be solved.
However, the following might be of interest if you want to use Bitmaps
with RGB_565 color space.

I'm working on a project in which I also needed to transform between
android.graphics.bitmap and cv::Mat back in June. As I didn't see your
solution that time, I wrote a little converter from RGB565 Bitmap to
cv::Mat in BGR888 colorspace.

Basically I implemented two functions in C++ that change colorspace
from RGB565 (byte array) to BGR888 (cv::Mat) and back:

/
*******************************************************************************/

void RGB565_2_cvMat( Mat& ar_Mat, char* ap_Data, int ao_Length, int
ao_Height, int ao_Width)
{
CV_Assert ( ao_Length == (ao_Height * ao_Width) * 2 );

ar_Mat = Mat( ao_Height, ao_Width, CV_8UC3 );

CV_Assert( ar_Mat.data );

typedef cv::Vec<unsigned char, 3> ELEM3_t;

cv::MatIterator_<ELEM3_t> lo_Iter, lo_Iter_End;

lo_Iter = ar_Mat.begin<ELEM3_t>();
lo_Iter_End = ar_Mat.end<ELEM3_t>();

const uint16_t lc_RedMask = 0xF800;
const uint16_t lc_GreenMask = 0x07E0;
const uint16_t lc_BlueMask = 0x001F;

int lo_BytePos = 0;

for ( ; lo_BytePos < ao_Length && lo_Iter != lo_Iter_End ;
lo_BytePos += 2, ++lo_Iter )
{
// glue 2 bytes for the pixel
uint16_t lo_PixelVal =
static_cast<uint16_t>( static_cast<unsigned
char>( ap_Data[ lo_BytePos ] ) ) |
( static_cast<uint16_t>( static_cast<unsigned
char>( ap_Data[ lo_BytePos + 1 ] ) ) << 8 );

uint8_t lo_RedVal = static_cast<uint8_t>((lo_PixelVal &
lc_RedMask) >> 11); // extracting color value
uint8_t lo_GreenVal = static_cast<uint8_t>((lo_PixelVal &
lc_GreenMask) >> 5); // - " -
uint8_t lo_BlueVal = static_cast<uint8_t>(lo_PixelVal &
lc_BlueMask); // - " -

lo_RedVal <<= 3; // expanding to 8 bit
lo_GreenVal <<= 2; // -"-
lo_BlueVal <<= 3; // -"-

(*lo_Iter)[0] = lo_BlueVal; // general OpenCV functions (e. g.
imread, imwrite) expect cv::Mat with BGR 888 color model
(*lo_Iter)[1] = lo_GreenVal; // -"-
(*lo_Iter)[2] = lo_RedVal; // -"-

}

CV_Assert( lo_BytePos == ao_Length && lo_Iter == lo_Iter_End ); //
assure we have reached the end of both, byte array and cv::Mat...
}

int cvMat_2_RGB565_getLengthByteArray (cv::Mat& ar_Mat)
{
return (ar_Mat.cols * ar_Mat.rows) * 2;
}

void cvMat_2_RGB565(Mat& ar_Mat, char* ap_Data, int ao_Length, int
ao_Height, int ao_Width)
{
CV_Assert( ar_Mat.data );
CV_Assert( ao_Height == ar_Mat.rows );
CV_Assert( ao_Width == ar_Mat.cols );
CV_Assert( ao_Length == (ar_Mat.cols * ar_Mat.rows) * 2);

typedef cv::Vec<unsigned char, 3> ELEM3_t;

cv::MatIterator_<ELEM3_t> lo_Iter, lo_Iter_End;

lo_Iter = ar_Mat.begin<ELEM3_t>();
lo_Iter_End = ar_Mat.end<ELEM3_t>();

int lo_BytePos = 0;

for ( ; lo_BytePos < ao_Length && lo_Iter != lo_Iter_End ;
lo_BytePos += 2, ++lo_Iter )
{
// glue 2 bytes for the pixel
uint16_t lo_PixelVal =
static_cast<uint16_t>( static_cast<unsigned char>( (*lo_Iter)[0] ) >>
3 ) | ( static_cast<uint16_t>( static_cast<unsigned char>( (*lo_Iter)
[1] ) >> 2 ) << 5 ) | ( static_cast<uint16_t>( static_cast<unsigned
char>( (*lo_Iter)[2] ) >> 3 ) << 11 );

ap_Data[ lo_BytePos + 1 ] = static_cast<unsigned
char>((lo_PixelVal & 0xFF00) >> 8);
ap_Data[ lo_BytePos ] = static_cast<unsigned char>((lo_PixelVal
& 0x00FF));
}

CV_Assert( lo_BytePos == ao_Length && lo_Iter == lo_Iter_End ); //
assure we have reached the end of both, byte array and cv::Mat...
}


/
*******************************************************************************/

The Java-Code that calls into these functions you can see as part of
the SWIG interface file here:

/
*******************************************************************************/

%pragma(java) moduleimports=%{
import com.opencv.jni.Mat; //import Mat wrapper

import java.nio.ByteBuffer;
import android.graphics.Bitmap;
%}


%pragma(java) modulecode=%{

/**
* Parses a cv::Mat to a android.graphics.Bitmap.
*
* The Mat must be in BGR_888 color space (common in OpenCV)
* The Bitmap will be in RGB_565 color space (Android renders GUI in
RGB_565)
*/
public static Bitmap cvMat2Bitmap ( Mat ao_Mat )
{
Bitmap lo_Bitmap = null;
ByteBuffer lo_Buf = null;
byte[] la_Bytes = null;
int lo_Length = 0;
int lo_Width = 0;
int lo_Height = 0;

lo_Width = ao_Mat.getCols();
lo_Height = ao_Mat.getRows();

lo_Length = cvMat_2_RGB565_getLengthByteArray(ao_Mat);

la_Bytes = new byte[ lo_Length ];

cvMat_2_RGB565(ao_Mat, la_Bytes, lo_Length, lo_Height, lo_Width );

lo_Buf = ByteBuffer.wrap(la_Bytes);

lo_Bitmap = Bitmap.createBitmap(lo_Width, lo_Height,
Bitmap.Config.RGB_565);
lo_Bitmap.copyPixelsFromBuffer(lo_Buf);

la_Bytes = null;

return lo_Bitmap;
}

/**
* Parses a android.graphics.Bitmap to a cv::Mat.
*
* The Bitmap must be in RGB_565 color space (Android renders GUI in
RGB_565)
* The Mat will be in BGR_888 color space (common in OpenCV)
*/
public static Mat Bitmap2cvMat ( Bitmap ao_Bitmap ) throws
IllegalArgumentException
{
if ( Bitmap.Config.RGB_565 != ao_Bitmap.getConfig() )
{
throw new IllegalArgumentException("Bitmap must be RGB_565 ");
}

ByteBuffer lo_Buf = null;
byte[] la_Bytes = null;
int lo_Width = 0;
int lo_Height = 0;
Mat lo_Mat = new Mat();

lo_Width = ao_Bitmap.getWidth();
lo_Height = ao_Bitmap.getHeight();

la_Bytes = new byte[ lo_Height * lo_Width * 2 ];

lo_Buf = ByteBuffer.wrap(la_Bytes);

ao_Bitmap.copyPixelsToBuffer(lo_Buf);

RGB565_2_cvMat(lo_Mat, la_Bytes, la_Bytes.length, lo_Height,
lo_Width );

return lo_Mat;
}
%}

%include "various.i"
%include "arrays_java.i"

%apply (char* BYTE) { char* }; //byte[] to char *

/
*******************************************************************************/


Afterwards in your Java-Code you only need to call


yourMat = yourSWIG_ClassName.Bitmap2cvMat(yourBitmap);

Then manipulate yourMat as you wish and call


yourBitmap = yourSWIG_ClassName.cvMat2Bitmap(yourMat);

to get it back as Bitmap in order for showing it in an ImageView or
whatever.


I extracted this part of my project to a little demo project, that
loads a bitmap from the apps resources, transforms it to cv::Mat,
calls cv::putText() on the cv::Mat, transforms it back to Bitmap and
shows it. If anyone is interested in the entire demo project, plz send
me a message and I'll send you the code..

Maybe this is helpful for anybody. Keep doing such a good work!

Wolfram

[Aaron Brown]

Welcome to the list, Wolfram! And thanks for posting your code!

Noam: I haven't had any trouble converting the Bitmaps from my
JPEGtoRGB888 method to Mats and then back again.

Once they're in RGB888 format, the conversion functions in
org.opencv.android worked fine for me.

Are they not working for you? If not, what behavior are they giving
you?

-Aaron

[kelly Jung]

Hi
Actually i had a problem with bitmapToMat method.
I didn't know that this method work with only ARGB_8888
thans to Aaron Brown, i solved my problem
thank u so much

In korea, information about android-openc is very insufficient
I don't know about other natio's situation
but you guys's comments are really helpful

thank u