Android: Draw offscreen Surface to onscreen Surface

[mrousavy]

Hey!

I'm building a Camera2 app which renders filters. I want to use Skia + SKSL for this, so I set up two android.view.Surfaces:

1. An offscreen surface I created with OpenGL's eglCreatePbufferSurface

2. An onscreen surface which I receive from the SurfaceView view, and then convert it to an OpenGL surface using eglCreateWindowSurface

I then use MakeFromBackendRenderTarget to wrap the OpenGL Surfaces to two Skia Surfaces, inputSurface and outputSurface.

Initialization runs fine and without any OpenGL errors, but I am not sure how I can draw the contents of the first surface (offscreen) into the second surface (onscreen) in my Frame Received Callback.

Also, I can draw stuff onto the first surface (offscreen) without anything crashing (I'm just not sure if it works as I can't see anything yet), but once I draw onto the second surface (onscreen) it crashes with the following error: "Unable to update texture contents"

Should I use one OpenGL context for everything?

Surely I can't just do something like:

// take snapshot of offscreen context
_inputSurface->flushAndSubmit();
auto image = _inputSurface->makeImageSnapshot();

// write that to onscreen context
_outputSurface->getCanvas()->drawImage(image, 0, 0);
// (do shader here)
_outputSurface->flushAndSubmit();

, right?

[Brian Salomon]

Hi, you probably should just use a single OpenGL context. Skia generally doesn't allow you to draw content from one OpenGL context to another (outside of some limited facilities that require the OpenGL contexts are made as "shared").

Skia does support çreating and rendering to offscreen surfaces. You just call SkSurface::MakeRenderTarget() and Skia will create an OpenGL texture wrapped in a SkSurface that you can draw to and then snap an image from (as in your example code). Note that if you're using a single GrDirectContext/OpenGL context you don't need the first flushAndSubmit. Skia ensures things on the same GrDirectContext happen in the right order.

Brian

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/c2286057-d667-45ec-a543-5b8ba995a61en%40googlegroups.com.

[mrousavy]

Thanks for your reply - from my understanding you cannot use one OpenGL context from two different threads though, can you?

My Camera Thread sends new Frames at 240 FPS, my UI Thread renders the latest Frame at 60 FPS (so some frames will be skipped).

[mrousavy]

I'm pretty sure I can use camerakit-android as a reference - they do everything in OpenGL. Unfortunately I am not very familiar with OpenGL, so I'm basically trying to convert this setup to Skia.

[Brian Salomon]

Ah, yeah, a single context won't work for you. I'm not an Android specialist by any means, but I believe you can have camera2 produce a SurfaceTexture. And then you can import that into the consumer's OpenGL context using this:

https://developer.android.com/ndk/reference/group/surface-texture#group___surface_texture_1ga0a204415e7f93b6b2a09cd4d71d3bb39

and finally import that texture into Skia with SkImage::MakeFromBackendTexture.

If there is a way to get an AHardwareBuffer with the image data then Skia has some facilities for importing those directly as either SkSurfaces or SkImages, but how to get from camera2 to AHardware buffer is definitely outside my knowledge.

I hope that helps.

Brian

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/aebc05d7-bc8c-4236-ab3e-a2bb840da986n%40googlegroups.com.

[mrousavy]

Hey!

I just picked up this project again, sorry for the delay here.

I found a way to get an AHardwareBuffer from my Camera2 pipeline, but I can't seem to create an SkImage from that. Here's my code:

#include <android/SkImageAndroid.h>

AHardwareBuffer* buffer = AHardwareBuffer_fromHardwareBuffer(jni::Environment::current(), hardwareBufferBoxed);
sk_sp<SkImage> image = SkImages::DeferredFromAHardwareBuffer(buffer, kOpaque_SkAlphaType);

For some reason this won't build:

ld: error: undefined symbol: SkImages::DeferredFromAHardwareBuffer(AHardwareBuffer*, SkAlphaType)

Are those sources not included in Skia by default? Is it maybe because of some min API level setting? Do I need to rebuild libskia.so with API level 26 or 28?

Thanks!

[mrousavy]

Ah yeah sorry I just saw the code - DeferredFromAHardwareBuffer is not available unless I compile against minSdk 26.

I think I just explored every possibility to be honest, there is no way to draw an android.media.Image to a Skia Surface. (unless I convert it to a byte[], but this surely will be way too slow for realtime use)