What is the correct way to set target resolution when display orientation is locked?

[Artem Matskevich]

Hello.

I use 1.0.0-alpha09 version of camera-core library.

I have activity with locked orientation:

<activity
 android:name=".camera.camerax.CameraActivity"
 android:configChanges="keyboardHidden|orientation|screenSize"
 android:screenOrientation="portrait" />

I use android.view.OrientationEventListener to set the target rotation (as ImageCapture.setTargetRotation method's doc proposes).

Here is how I create a capture use case:

private fun createCaptureUseCase(): ImageCapture {
 return ImageCapture.Builder()
 .setTargetName("Capture")
 .setTargetResolution(Size(1080, 1920))
 .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
 .build()
}

In this way I get photos with correct rotation, but with wrong resolution. If device is in natural portrait orientation then the photo resolution is 1080x1920 as expected. But if the device is rotated into landscape orientation then the resolution is 608x1080, not 1920x1080 as I exepcted to get. 

I've tried to comment setTargetRotation call, photo resolution was 1080x1920 regardless of device orientation, but photo rotation wasn't correct in landscape orientations.

I've also read ImageCapture.Builder.setTargetResolution docs (particularly the next statement):
The resolution Size should be expressed at the use cases's target rotation. For example, a device with portrait natural orientation in natural target rotation requesting a portrait image may specify 480x640, and the same device, rotated 90 degrees and targeting landscape orientation may specify 640x480.
but I can't change target resolution as capture usecase is already built and bound to camera at the moment when OrientationEventListener dispatches orientation updates and ImageCapture itself has no method for setting resolution.

Am I doing something wrong or there is a bug somewhere?

Thanks in advance.

[Charcoal Chen]

Thanks Artem. I have opened a bug to track this issue 148763432.

The root cause is when ImageCapture.Builder()#setTargetResolution is called, setTargetAspectRatioCustom() will also be set automatically to make the output image matching the aspect ratio of target resolution. But it is not calibrated when ImageCapture#setTargetRotation() is called. For a temporarily solution, you may be able to use ImageCapture#setTargetAspectRatioCustom() to make it match the orientation that you set by setTargetRotation(). We may check whether there is better way to make it transparent for the CameraX users.

Artem Matskevich 在 2020年1月31日 星期五上午2:45:12 [UTC+8] 的信中寫道：

[Artem Matskevich]

Thanks, Charcoel.

I've added the call to setTargetAspectRatioCustom after setTargetRotation and it solved the problem with photo resolution.
But there is one moment left I want to clarify.

I've overridden OrientationEventListener.onOrientationChanged in the following way:

   

    override fun onOrientationChanged(orientation: Int) {
        val rotation = orientation.orientationDegreesToSurfaceRotation()
        ...
    }

internal const val ROTATION_UNKNOWN = -1

internal fun Int.orientationDegreesToSurfaceRotation(): Int {
    return when (this) {
        in 0..29, in 330..359 -> Surface.ROTATION_0
        //in 60..119 -> Surface.ROTATION_90
        in 60..119 -> Surface.ROTATION_270
        in 150..209 -> Surface.ROTATION_180
        //in 240..299 -> Surface.ROTATION_270
        in 240..299 -> Surface.ROTATION_90
        else -> ROTATION_UNKNOWN
    }
}

I had to swap Surface.ROTATION_90 with Surface.ROTATION_270 for 60..119 range and Surface.ROTATION_270 with Surface.ROTATION_90 for 240..299 range for photo to be captured with correct rotation. I've tested this approach on several devices with various android versions and it gave me the correct photo rotation. If this is an intended behaviour can you explain why rotation 90 and 270 should be swapped?

понедельник, 3 февраля 2020 г., 12:05:27 UTC+3 пользователь Charcoal Chen написал:

[Charcoal Chen]

Please refer to the web doc for OrientationEventListener#onOrientationChanged(int) and Display#getRotatoin().

In OrientationEventListener#onOrientationChanged(int) --> orientation is 0 degrees when the device is oriented in its natural position, 90 degrees when its left side is at the top

In Display#getRotatoin() --> if the device is rotated 90 degrees counter-clockwise, to compensate rendering will be rotated by 90 degrees clockwise and thus the returned value here will be Surface#ROTATION_90.

For Display#getRotatoin(), device's right side will be at the top when rotating 90 degrees counter-clockwise.

Therefore, it will result in what you mentioned as the question.

Artem Matskevich 在 2020年2月7日 星期五下午6:58:07 [UTC+8] 的信中寫道：

[teddy boy]

Can you share the full setup? I tried your code and setTargetAspectRatioCustom(Rational(rotation, 1)) it always cropped photo resolution when the phone is landscape. Because setTargetAspectRatioCustom will pre-crop image. My device Nokia 7 Plus Android 10.

This is my code it worked fine without using setTargetAspectRatioCustom so output image will not be pre-cropped when using setTargetRotation

private fun getOrientationFromDegrees(orientation: Int): Int {
    return when {
        orientation == OrientationEventListener.ORIENTATION_UNKNOWN -> {
            Surface.ROTATION_0
        }
        orientation >= 315 || orientation < 45 -> {
            Surface.ROTATION_0 //portrait
        }
        orientation < 135 -> {
            //Surface.ROTATION_90
            Surface.ROTATION_270 //landscape
        }
        orientation < 225 -> {
            Surface.ROTATION_180
        }
        else -> {
            //Surface.ROTATION_270
            Surface.ROTATION_90
        }
    }
}

val orientationEventListener = object : OrientationEventListener(this) {
    override fun onOrientationChanged(orientation: Int) {
        imageCapture?.setTargetRotation(getOrientationFromDegrees(orientation))
    }
}.apply { enable() }


val rotation = viewFinder.display.rotation
val isLandscape = rotation == Surface.ROTATION_90 || rotation == Surface.ROTATION_270

imageCapture = ImageCapture.Builder()
    .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
    .setTargetResolution(if (isLandscape) Size(1920, 1080) else Size(1080, 1920))
    .setTargetRotation(rotation) 
    .build()

[Artem Matskevich]

Please confirm that I'm understanding right.

Device orientation from sensor (through OrientationEventListener) is measured clockwise, display rotation must compensate device rotation and hence display rotation is measured counter-clockwise.

And then degrees to rotation conversion should be modified in the following way:

internal fun Int.orientationDegreesToSurfaceRotation(): Int {
    return when (360 - this) {
        in 330..359, in 0..29 -> Surface.ROTATION_0
        in 60..119 -> Surface.ROTATION_90
        in 150..209 -> Surface.ROTATION_180
        in 240..299 -> Surface.ROTATION_270
        else -> ROTATION_UNKNOWN
    }
}

пятница, 7 февраля 2020 г., 14:46:25 UTC+3 пользователь Charcoal Chen написал:

[Charcoal Chen]

Hi Artem,

Below description is correct.

    --> Device orientation from sensor (through OrientationEventListener) is measured clockwise, display rotation must compensate device rotation and hence display rotation is measured counter-clockwise.

But the code in your last reply is incorrect. The correct one should be one you replied previously that Surface.ROTATION_90 and Surface.ROTATION_270 need to be swapped.

Artem Matskevich 在 2020年2月7日 星期五下午11:31:29 [UTC+8] 的信中寫道：

[Artem Matskevich]

Hi Charcoel,

Is code incorrect even considering that I've changed angle measurement direction: when (360 - this)?

понедельник, 10 февраля 2020 г., 14:47:34 UTC+3 пользователь Charcoal Chen написал:

[Charcoal Chen]

Sorry, I missed "when (360 - this)".

There might be a little problem when the coming orientation value is exactly 0.
"360 - 0" is not in the range for Surface.ROTATION_0 in your code.

For the others, I think the code can work fine.

Artem Matskevich 在 2020年2月11日 星期二上午1:10:50 [UTC+8] 的信中寫道：

[Artem Matskevich]

Yes, of course, I forgot the case when angle is 0.

Thank you very much for your help, Charcoel!

вторник, 11 февраля 2020 г., 7:23:50 UTC+3 пользователь Charcoal Chen написал: