ImageReader's onImageAvailable isn't called anymore because JNIEnv is null

[Peter Retzlaff]

Hello,

I'm developing a screen recording app using the MediaProjection API added in API level 21 (which doesn't require using the NDK, but bear with me here). It works great most of the time. However, sometimes it will just stop recording (meaning that no more video frames are coming in). As of now, I couldn't reproduce this issue on one of my devices, but I know for sure that it happens for some of my users. I managed to narrow the problem down to the fact, that after some time the 'onImageAvailable' callback isn't called anymore. However, I still need to figure out, why that is.

This is how I set up the media projection (this should all be pretty standard stuff):

public void onActivityResult(int requestCode, int resultCode, Intent data) {
        // get media projection
        mMediaProjection = mProjectionManager.getMediaProjection(resultCode, data);
        mMediaProjection.registerCallback(new MediaProjectionCallbackLogger(), null);

        // get DPI
        final DisplayMetrics metrics = new DisplayMetrics();
        final Point size = new Point();
        mActivity.getWindowManager().getDefaultDisplay().getRealMetrics(metrics);
        mActivity.getWindowManager().getDefaultDisplay().getRealSize(size);
        mScreenDensity = metrics.densityDpi;
        mDisplayWidth = size.x / 2;
        mDisplayHeight = size.y / 2;

        // create surface
        mImageReader = ImageReader.newInstance(mDisplayWidth, mDisplayHeight, PixelFormat.RGBA_8888, 2);

        // create virtual display writing to surface
        mVirtualDisplay = mMediaProjection.createVirtualDisplay("MyVirtualDisplay",
                mDisplayWidth, mDisplayHeight, mScreenDensity, DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
                mImageReader.getSurface(), new VirtualDisplayCallbackLogger(), null /*Handler*/);

        mImageReader.setOnImageAvailableListener(new ImageAvailableListener(), mHandler);
}

And here is the structure of my ImageAvailableListener implementation:

private class ImageAvailableListener implements ImageReader.OnImageAvailableListener {
        @Override
        public void onImageAvailable(ImageReader reader) {
            Image img = reader.acquireLatestImage();

            if (img != null) {
                // Write image data to a bitmap, which is then rendered to a video.
                // [...]
                img.close();
            }
        }
}

As I said, this callback should be called each time a new frame is available and this works fine for some time. But then it just stops being called, which probably means that either the virtual display stops producing frames (which would be odd, as it is neither stopped nor resumed) or there is some other condition under which the callback isn't called by the system.

So I looked at ImageReader.java to find out if there is such a condition. The callback is called in the 'handleMessage' implementation of ImageReader.ListenerHandler, which in turn gets triggered from inside ImageReader.postEventFromNative().

This here is the native code that calls 'postEventFromNative':

void JNIImageReaderContext::onFrameAvailable(const BufferItem& /*item*/)
{
    ALOGV("%s: frame available", __FUNCTION__);
    bool needsDetach = false;
    JNIEnv* env = getJNIEnv(&needsDetach);
    if (env != NULL) {
        env->CallStaticVoidMethod(mClazz, gImageReaderClassInfo.postEventFromNative, mWeakThiz);
    } else {
        ALOGW("onFrameAvailable event will not posted");
    }
    if (needsDetach) {
        detachJNI();
    }
}

And this is the implementation of getJNIEnv():

JNIEnv* JNIImageReaderContext::getJNIEnv(bool* needsDetach) {
    LOG_ALWAYS_FATAL_IF(needsDetach == NULL, "needsDetach is null!!!");
    *needsDetach = false;
    JNIEnv* env = AndroidRuntime::getJNIEnv();
    if (env == NULL) {
        JavaVMAttachArgs args = {JNI_VERSION_1_4, NULL, NULL};
        JavaVM* vm = AndroidRuntime::getJavaVM();
        int result = vm->AttachCurrentThread(&env, (void*) &args);
        if (result != JNI_OK) {
            ALOGE("thread attach failed: %#x", result);
            return NULL;
        }
        *needsDetach = true;
    }
    return env;
}

Sooo... there is a case in which the callback doesn't get called: If getJNIEnv() returns NULL, which only happens if attaching the current thread fails. As I said, I can't reproduce the problem, so I can't confirm that this is actually what happens. But I'm assuming it for now.

My question now is: What could lead to the current thread not being able to attach to the Java VM all of a sudden? Apparently it worked before, so what could have changed? And more generally, under which circumstances does a call to 'AttachCurrentThread' fail? I was unable to find any in-depth documentation regarding the possible error codes and/or failure cases..

I am open to any hints and wild guesses as to what is happening here.

Thanks,

Peter

[Praveen Pandey]

What was the reason for this? Have you resolved it.