Camerax orietation problem in multiple devices

[Shivani Patel]

Hi,

Rotated images appeared on several devices when I was using the camerax 1.3.1 stable library. Here are some details regarding the problem.

Device Name: Zebra ET45 (OS: 11)

Device Name: Nokia G21

Device Name: OPPO - CPH2205 (OS: 13)

Device Name: OPPO CPH2239 (OS: 11)

We are doing operations according to the orientation of the device and EXIF information.

[Xi Zhang (张熹)]

Thank you for reporting the issue Shivani. Could you share a minimal reproducible code with us? For example, what are the UseCases used by your app? I assume that you are seeing this in Preview.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/02d11b83-cdbc-49a7-a570-491ca44c2c9en%40android.com.

[Shivani Patel]

In the preview, we see the perfect image but after capturing it is rotating.

Here is the code snippet

Before the issue, we have this logic

private int rotation_; // orientation of device

private boolean rotateImage; // It is enabled when we get orientation from Exif info

Matrix mat = new Matrix();
LogUtils.LOGI("still", "functionRotation" + rotation_);
try {
if (rotateImage) {
if (rotation_ != 0) {
mat.postRotate(rotation_);
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
} else {
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
}
} else {
mat.postRotate(rotation_ - 90);
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
}
} catch (Exception e) {
BugfenderUtils.e("compressImageResult1", e);
}

After we got issues in multiple devices we have implemented this code

Matrix mat = new Matrix();
BugfenderUtils.v(TAG, "functionRotation: " + rotateImage + " " + rotation_);
try {
if (rotateImage) {
if (rotation_ != 0) {
mat.postRotate(rotation_);
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
} else {
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
}
} else {
String deviceModel = Utils.getDeviceModel();

if (!TextUtils.isEmpty(deviceModel) &&
(deviceModel.equalsIgnoreCase(Constant.NOKIA_G21_MODEL) ||
deviceModel.equalsIgnoreCase(Constant.OPPO_CPH2205_MODEL) ||
deviceModel.equalsIgnoreCase(Constant.OPPO_CPH2239_MODEL))) {
mat.postRotate(rotation_ - 90);
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
} else {
if (isCapturedImage && orientation == ExifInterface.ORIENTATION_UNDEFINED) {
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
} else {
mat.postRotate(rotation_ - 90);
scaledBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), mat, true);
}
}
}
} catch (Exception e) {
BugfenderUtils.e("compressImageResult1", e);
}

[Xi Zhang (张熹)]

Are you displaying an ImageProxy? In that case, instead of checking the device rotation and Exif info, could you try always using the value of imageProxy.getImageInfo().getRotationDegrees()?

[Shivani Patel]

Thanks for the suggestion, let me check this 

[Xi Zhang (张熹)]

I see. In locked orientation, CameraX can no longer get the device orientation. It's recommended to set the target rotation using motion sensor readings. For example, using the RotationProvider API. If you are using the CameraController API, CameraX will take care of this for you. After that, you should be able to get the correct rotation degrees via imageProxy.getImageInfo().getRotationDegrees().

On Wed, Feb 14, 2024 at 9:11 PM Shivani Patel <shivanip...@gmail.com> wrote:

Hi,

I have checked the rotation degree in the image proxy but I am getting a 90 value for both portrait and landscape(without rotating activity and view), In this case how we can rotate the landscape image to portrait because we are getting the same value(90) in both the scenario. 

[Shivani Patel]

Hi,

I have implemented it as per your suggestion but getting the below exception 

androidx.camera.core.ImageCaptureException: Not bound to a valid Camera [ImageCapture:androidx.camera.core.ImageCapture-184b3b15-6d6e-4726-b32c-21209ce728ea]
                  at androidx.camera.core.ImageCapture.sendInvalidCameraError(ImageCapture.java:1060)
                  at androidx.camera.core.ImageCapture.takePictureInternal(ImageCapture.java:1041)
                  at androidx.camera.core.ImageCapture.takePicture(ImageCapture.java:719)
                  at androidx.camera.view.CameraController.takePicture(CameraController.java:718)
                  at com.ebestiot.ircamera.camera.ui.CameraXActivity.lambda$setListener$0(CameraXActivity.java:321)
                  at com.ebestiot.ircamera.camera.ui.CameraXActivity.$r8$lambda$9VJSjTFJVIpW-tmXM15hMVbLAj8(Unknown Source:0)
                  at com.ebestiot.ircamera.camera.ui.CameraXActivity$$ExternalSyntheticLambda3.onClick(Unknown Source:2)
                  at android.view.View.performClick(View.java:7892)
                  at android.view.View.performClickInternal(View.java:7869)
                  at android.view.View.-$$Nest$mperformClickInternal(Unknown Source:0)
                  at android.view.View$PerformClick.run(View.java:30891)
                  at android.os.Handler.handleCallback(Handler.java:942)
                  at android.os.Handler.dispatchMessage(Handler.java:99)
                  at android.os.Looper.loopOnce(Looper.java:226)
                  at android.os.Looper.loop(Looper.java:313)
                  at android.app.ActivityThread.main(ActivityThread.java:8762)
                  at java.lang.reflect.Method.invoke(Native Method)
                  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:604)
                  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1067)

[Xi Zhang (张熹)]

What's your code look like? If you are using Camera controller , did you bind it to lifecycle?

[Shivani Patel]

This is my code and it is in Activity

mController = new LifecycleCameraController(this);

mController.bindToLifecycle(this);

mBinding.preview.setController(mController);

// Create a provider.

RotationProvider mRotationProvider = new RotationProvider(this);

// Add listener to receive updates.

mRotationProvider.addListener(executor, rotation -> {

    if (mImageCapture != null) {

        mImageCapture.setTargetRotation(rotation);

    }

});

    mImageCapture = builder

             //images may capture faster but the image quality may be reduced.

              .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)

              .setTargetResolution(new Size(Constant.TARGET_HEIGHT,

                                Constant.TARGET_WIDTH))

               .build();

cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalysis,

        mImageCapture);

[Xi Zhang (张熹)]

CameraController is a higher level replacement of the cameraProvider API. So you don't want to use both. If you are using CameraController then you don't need to create any UseCases yourself. You can take a look at our code sample here: https://github.com/androidx/androidx/blob/androidx-main/camera/integration-tests/viewtestapp/src/main/java/androidx/camera/integration/view/CameraControllerFragment.java

If you are not using CameraController, then you can just remove the following code:

mController = new LifecycleCameraController(this);

mController.bindToLifecycle(this);

mBinding.preview.setController(mController);

[Shivani Patel]

Hi,

I have removed the controller code but below is the result (I have tested it in Samsung s20 FE)

1. Portrait image rotated to landscape 

2. When I am rotating the device to landscape getting the below exception

                 java.util.concurrent.RejectedExecutionException: Task androidx.camera.view.RotationProvider$ListenerWrapper$$ExternalSyntheticLambda0@7384301 rejected from java.util.concurrent.ThreadPoolExecutor@b2505a6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5]
                  at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2072)
                  at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:834)
                  at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
                  at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:644)
                  at androidx.camera.view.RotationProvider$ListenerWrapper.onRotationChanged(RotationProvider.java:186)
                  at androidx.camera.view.RotationProvider$1.onOrientationChanged(RotationProvider.java:109)
                  at android.view.OrientationEventListener$SensorEventListenerImpl.onSensorChanged(OrientationEventListener.java:163)
                  at android.hardware.SystemSensorManager$SensorEventQueue.dispatchSensorEvent(SystemSensorManager.java:991)
                  at android.os.MessageQueue.nativePollOnce(Native Method)
                  at android.os.MessageQueue.next(MessageQueue.java:335)
                  at android.os.Looper.loopOnce(Looper.java:186)

                  at android.os.Looper.loop(Looper.java:313)
                  at android.app.ActivityThread.main(ActivityThread.java:8762)
                  at java.lang.reflect.Method.invoke(Native Method)
                  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:604)
                  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1067)

[Xi Zhang (张熹)]

Let me get one of the devices and test it. Please stay tuned.

[Shivani Patel]

Could you please update me on this?

[Xi Zhang (张熹)]

So I tested our test app on the Oppo A54 CPH2239. The behavior is working as expected. That is, if we set the target rotation correctly, we can get an "upright" image by applying the value of ImageInfo#getRotationDegrees() to the Bitmap.  To resolve the RejectedExecutionException you get, did you forget to call RotationProvider#removeListener when the UI lifecycle ends? You would get that error when the rotation changes and the executor is no longer valid. 

If this is still not working, you can upload a minimal reproducible example to GitHub and I will be happy to debug it for you.

[Shivani Patel]

Hi,

I have tried the above logic and it is working when we have orientation data in Exif Info but it is not working when we have "undefined" orientation.

We are getting "undefined" orientation for many devices. 

[Xi Zhang (张熹)]

Could you name a device that I can reproduce the issue with?

[Shivani Patel]

Hi,

These are the some devices in which we are getting "undefined" orientation 

HONOR X9a 5G

Galaxy A22 5G (a22x) (SM-A226BR)

moto g22 (hawaiip) (moto g22)

A93 (OP4C51L1) (CPH2123)

HONOR 20/Nova 5T (HWYAL) (YAL-L21)

realme GT NEO 3 (RED8BEL1) (RMX3561)

Redmi 10C (fog) (220333QAG)

moto g(20) (java) (moto g(20)) 

[Xi Zhang (张熹)]

I tested it on moto g22 with our test app, and the result is still correct. You can check out the source code of the displayImage() function here: https://github.com/androidx/androidx/blob/d8452c269374b0a4f68ade17e0b2fd9507197ff5/camera/integration-tests/viewtestapp/src/main/java/androidx/camera/integration/view/CameraControllerFragment.java#L561

Otherwise, if you can upload a minimal reproducible example to GitHub, I can take a look at it.