NDK and access to native Skia Bitmap functions

[Eric]

Per pixel operations can really benefit from being done in native code
but will it be possible with the NDK to get access to the SkBitmap
representation of the Java Bitmap object and then use the SkBitmap
functions to access the pixel data?

I am asking because the current way to manipulate pixels in Android
from Java is to call the expensive getPixels and setPixels methods. It
would be great to take these out of the equation so that there is no
more converting back and forth between jintArrays and the entire
operation can be done in native code.

[Jack Palevich]

Yeah, that's a great idea, and we hope to provide such an API in a future release.

There's an open feature request for it in our internal bug database, but it didn't make the cut for Cupcake.

In the meantime, please consider using Bitmap.copyPixelsToBuffer as a faster alternative to Bitmap.getPixels and consider using BitmapFactory.decodeByteStream with the easy-to-construct BMP file format as a faster alternative to Bitmap.setPixels.

Also, for the special circumstance of loading a Bitmap as an OpenGL texture, consider using GLUtils.texImage2D.

[Eric]

Thanks, great to hear that it is on the list for the future.

One other benefit of direct native access to the pixel data of a
bitmap other than speed is memory. I currently can't use methods you
suggested because for large bitmaps it will take up too much memory,
the getPixels setPixels way does have one advantage and that is that
you can process it in chunks. But with direct access, memory would not
be an issue. The developer would only have to deal with the different
bitmap configs (unless by design the developer chooses to only pass
ARGB_8888 bitmaps through the JNI )

Now that this won't make it in the initial NDK the only way to do fast
per pixel operations is to provide a Java buffer and then call
getIntArrayElements and ReleaseIntArrayElements in your native code.
Is it more common for the VM to simply return a pointer to the actual
data, or is it more likely that the VM will allocate memory and make a
copy? If it only makes a copy under rare conditions then doing it
this way won't be so bad.

On Apr 15, 8:10 pm, Jack Palevich <jack...@google.com> wrote:
> Yeah, that's a great idea, and we hope to provide such an API in a future
> release.
>
> There's an open feature request for it in our internal bug database, but it
> didn't make the cut for Cupcake.
>
> In the meantime, please consider using Bitmap.copyPixelsToBuffer as a faster
> alternative to Bitmap.getPixels and consider using
> BitmapFactory.decodeByteStream with the easy-to-construct BMP file format as
> a faster alternative to Bitmap.setPixels.
>
> Also, for the special circumstance of loading a Bitmap as an OpenGL texture,
> consider using GLUtils.texImage2D.
>

[Jack Palevich]

Oh, there's also a Bitmap.copyPixelsFromBuffer that I hadn't noticed before. Calling that would probably be faster than decodeByteStream.

If you can't copy the whole Bitmap at once due to memory issues, you could consider using a smaller auxiliary Bitmap object and the Canvas.drawBitmap operator to copy the portions of the large canvas into the smaller canvas.

The big advantage of using copyPixels instead of getPixels is that the pixels are kept in their native format, rather than being expanded to/from 32-bits-per-pixel.

It's not my area of expertise, but I am under the impression that our VM will currently always make a copy of the array contents when you call GetXXXArrayElements.

If you're just working on a portion of a large array, you might want to call Get/Set<type>ArrayRegion instead.

[fadden]

On Apr 15, 3:12 pm, Jack Palevich <jack...@google.com> wrote:
> It's not my area of expertise, but I am under the impression that our VM
> will currently always make a copy of the array contents when you call
> GetXXXArrayElements.
>
> If you're just working on a portion of a large array, you might want to call
> Get/Set<type>ArrayRegion instead.

The Get<type>ArrayElements calls *may* copy the data. In the current
implementation, they don't, but if we move to a compacting GC this may
change. (There's a special CheckJNI mode in cupcake that forces a
copy, to test for code that unwittingly expects it to work one way or
the other.)

The Get/Set<type>ArrayElements calls *always* copy the data. This is
great if you actually want a copy of the data, not so great if you
were hoping to just modify it in place. As Jack mentioned, this may
have some advantages if you only need to work on a small piece of a
large array.

Some notes here:

http://android.git.kernel.org/?p=platform/dalvik.git;a=blob_plain;f=docs/jni-tips.html;hb=HEAD#RegionCalls

[Eric]

Thanks for the info, I am familiar with the Android JNI Tips document.
So currently because no copy is made it wouldn't be too bad for
performance to use Java arrays as an in between step, but one should
of course not rely on this for the future.

In the end, I would love to use the NDK to do be able to do something
like the following:

static bool adjustPixels_ARGB_8888(JNIEnv* env, const SkBitmap*
bitmap,
int x, int y, int width, int height)
{
SkAutoLockPixels alp(*bitmap);
void* src = bitmap->getPixels();
if (NULL == src|| bitmap->config() != SkBitmap::kARGB_8888_Config)
{
return false;
}

src = bitmap->getAddr(x, y);
for (int y = 0; y < height; y++) {
SkPMColor* p = (SkPMColor*)src;
for (int x = 0; x < width; x++) {
// adjust *p
p++;
}
src = (char*)src + bitmap->rowBytes();
}

return true;
}

But this would require the skia lib and public access to the native
bitmap field of a Bitmap object

>  http://android.git.kernel.org/?p=platform/dalvik.git;a=blob_plain;f=d...

[David Turner]

Just for the record, direct access to the Skia lib is *not* planned for any revision of the NDK.
(The Skia headers are changing constantly, and there is no way we're going to provide a stable ABI for this lib).

We'd rather provide a simple C-based API which would allow you to do something similar
with more generic data structure.

[Eric]

Having access to the Skia lib just for altering bitmap pixels directly
would indeed be a bit too much.

Thinking out loud here for a future solution. Although the OpenGL lib
is not yet accessible a lot of the Java OpenGL functions require a
java.nio.Buffer object. Can these Buffer objects be safely accessed
for read/write in native code using the GetDirectBufferAddress JNI
function without a performance hit?

If so, you could provide a similar function in android.graphics.Bitmap
that creates a Bitmap object from a user supplied java.nio.Buffer:

public static Bitmap createPointerBasedBitmap(Buffer pointer,
int stride, int width, int height, Bitmap.Config config)

It would not copy the data from the buffer and like the gl*Pointer()
functions it must be a direct buffer allocated on the native heap.

Developers can then do what they want with the pixels in native code
and use it like a normal Bitmap object back in Java. This would not
require any additional C-based API and I think Skia bitmaps can
already handle user-specified buffers internally.

Just a suggestion, not sure if it is feasible, any thoughts?

On Apr 16, 4:32 pm, David Turner <di...@android.com> wrote:
> Just for the record, direct access to the Skia lib is *not* planned for any
> revision of the NDK.
> (The Skia headers are changing constantly, and there is no way we're going
> to provide a stable ABI for this lib).
>
> We'd rather provide a simple C-based API which would allow you to do
> something similar
> with more generic data structure.
>

[jcfrei]

I've been trying to implement some sort of native setPixel function for Android - is this in any way possible? Or do I read correctly that this is in fact not possible? If so, how was the bitmap-plasma from the samples implemented? they seem to be taking a bitmap from java and editing it in native mode.

[Tim Mensch]

It would likely be useful to look at the bitmap-plasma sample source to
figure out how it was implemented...just saying...

Or you could search the STABLE-APIS.html doc for jnigraphics.

Tim

[Chris Stratton]

On Monday, July 25, 2011 5:42:04 PM UTC-4, jcfrei wrote:

I've been trying to implement some sort of native setPixel function for Android - is this in any way possible? Or do I read correctly that this is in fact not possible? If so, how was the bitmap-plasma from the samples implemented? they seem to be taking a bitmap from java and editing it in native mode.

The most obvious difference between today and the history of this thread is that there now are native interfaces to opengl.